Who Will Own the Inference Control Plane?

Previously in the series: The missing control plane must connect enterprise intent to the models, runtimes, hardware, regions, and providers that execute it.

Every incumbent in this stack sells something other than the decision. Clouds, neoclouds, and managed inference providers sell capacity; Fireworks, Baseten, and Together alone raised about $3.8 billion in under four weeks this summer.²⁸ Amazon Bedrock’s prompt routing chooses between exactly two models of the same family.²⁹ Labs sell models. OpenRouter routes across rented providers and takes a fee on what you rent.³⁰ Observability tools report on the fleet but do not change it. NVIDIA sells silicon, and Dynamo, the free, open-source software it calls the operating system of AI factories, is built to get the most out of NVIDIA hardware.³¹ An optimizer built by any of them will, at the margin, route toward its owner. Only an outside layer, whose sole product is the decision, sees how the same workload behaves across all of them.

The other objection is that this is just cloud cost management. NetApp bought Spot.io, which managed cloud compute purchasing for its customers, for a reported $450 million in 2020. In 2025 it sold Spot, with CloudCheckr, to Flexera for a reported $100 million.³² The reasons were partly NetApp’s, but a layer that only optimizes how a commodity is bought is always one feature release away from the seller building it in.

Inference differs in three ways. The optimization spans engine, cache, parallelism, and fabric, which no single vendor’s console sees end to end. Quality is in the loop: a change that saves money can break outputs, so what the customer buys is a validated change. And the decision spans vendors, so none of them can absorb it without losing what makes it useful. Spot’s buyer was the FinOps team. The buyer here is a practice that is still forming (call it inference engineering), and whether it grows out of platform engineering, FinOps, or ML infrastructure is one of the open questions of the next two years.

Replit built its router in-house.¹⁹ Some of the next ten companies at its scale will too. Even teams that have built their own tell us that keeping it tuned takes a meaningful share of their infrastructure headcount, and that they would buy if the gain were proven on their own traffic. And routing is only the visible part of this category; much of the recoverable cost sits beneath the router, in how the fleet is configured, placed, and validated as models and traffic change.

That is how infrastructure categories start: sophisticated operators build first, and the category forms when the next wave, which cannot staff a dedicated team, needs the same capability. AI-native companies with high-volume, predictable workloads move first, because the return lands in gross margin. Regulated enterprises follow, for placement, auditability, and sovereignty. Low-volume, bursty, or undifferentiated workloads stay on rented APIs, and should. The outcome is a portfolio: control the workloads where economics, quality, latency, or compliance justify it, and rent the rest.

Three questions follow, and we hear each of them regularly.

If I run a large enterprise, why own inference at all? Own the decisions first. A bank or a payroll processor does not need to own every GPU. It needs an inference engineering practice, the way it built DevOps and then FinOps. That team owns SLOs per workload, cost per task, and what runs where. It runs a fleet only where the decision rule calls for one: high volume, stable traffic, sensitive data, regulated placement.

If I am a startup that grew up on Bedrock or Baseten, when do I take ownership? When inference becomes one of the largest lines in cost of goods sold. When traffic is predictable enough to reserve. When a fine-tuned model’s weights have become an asset. When a provider’s price change or model retirement has hurt you once. When a latency SLO cannot be met on a shared endpoint. Start with a hybrid: one workload on owned or reserved capacity, behind the same control plane that decides where the rest runs. Moving the next one becomes a configuration change.

If I am a founder, when should I talk to Team8? When you are building any layer of the control plane described here, or the layers around it: model allocation, placement that treats power, region, and sovereignty as constraints, and evaluation that lets production change safely. We also want to hear from founders one layer down: the fabric that connects the fleet, the power and cooling that constrain it, and idle capacity that could be put to work. We would rather hear about it before the category has a name.

Inference will become a strategic enterprise system before it becomes a commodity utility. The most valuable companies in this market will sit between enterprise intent and infrastructure reality, translating business requirements into model and hardware choices and optimizing a distributed fleet against each workload’s SLO.

If we are right, by the end of 2028 the AI-native companies with the largest inference bills will run hybrid fleets behind a control plane they own. Swapping the model behind a production workload will be a configuration change rather than a rewrite, and the serving configuration beneath it will change more often than the application does. If inference is still bought the way API credits are bought today, we were wrong.

The next competitive advantage will not come from access to intelligence, which everyone will have, but from controlling how it is deployed: the models, the data, and the fleet that make it yours.