AI Workloads at the Telco Edge
Why Workload Placement is the Only Edge Strategy That Matters.
The telco industry is currently undergoing a structural transformation that goes far beyond a simple upgrade in connectivity speeds. For decades, the network was viewed primarily as a “pipe” something that moved data from point A to point B. Now, that paradigm has collapsed completely. Today, the physical infrastructure of the telecom world is being repurposed to serve as the foundational bedrock for AI.
Yet, if you open almost any telecom strategy report published in the last year and a half, you will find complicated diagrams with boxes labeled as the cloud, operator edge, network edge, local edge, and device edge. These diagrams create an assumption that simply moving closer to the “edge” will automatically improve AI performance. But is that even correct? For most of today’s AI, the biggest delay isn’t the time it takes for data to travel across the network, it is the time the model actually takes to think - the model reasoning time.
Because the actual computing power is the real bottleneck, the conversation cannot just be about network topology.
It has to shift toward economics, specifically the challenge of Edge Monetization. A recent report I was reading highlighted exactly this, recommending that operators need to be “selective,” to “run edge like a product,” and to “partner rather than compete” with hyperscalers.
This creates a situation where three major strategic dilemmas collide:
First is the technical problem ie understanding those different “edge” boxes, where they are placed, and how much computing capacity actually needs to be deployed.
Second is the financial problem meaning figuring out how to monetize this setup and determining who ultimately pays for that expensive capacity.
Third is the partnership problem which means deciding who to team up with and knowing when to build your own solutions versus buying existing ones.
These are all valid questions, but are they the most interesting ones? More importantly, they are not the questions an engineer, a product manager, or a founder building on top of telco infrastructure needs answered before making a real, capital-intensive decision.
To understand what is actually happening beneath the surface of the industry, we have to look past the marketing slogans and focus on the unglamorous engineering reality. The core challenge isn’t whether AI will be important. it’s the physical and economic question of where exactly the compute must live. This is not a binary choice between “The Cloud” and “The Edge.” Instead, it is a complex, tiered hierarchy of locations, each with its own set of strict rules regarding power, cooling, cost, and physics.
This deep dive is structured around exactly that shift. from “where should telcos play” to “how should AI workloads actually be placed, operated, and paid for.” The goal is to give operators, partners, and founders a way to reason about individual workloads with enough precision to make a real infrastructure decision.
Table of Content -
The Problem with the "Edge" Narrative
Why Workload Placement is Everything
The Edge is a Hierarchy, Not a Place - The New AI Architecture
The Four Classes of AI Workloads
The Engineering Playbook - How to Actually Build This
Where the Edge Budget Actually Is
A Five-Phase Plan Execution Playbook
Moving Beyond the Hype
What This Means for Founders
1. The Problem with the “Edge” Narrative
The mainstream telco narrative circles around three unresolved questions.
how deeply to invest in edge AI infrastructure,
who pays for it, and
where to partner rather than compete.
Cloud is treated as hyperscaler territory where telcos have essentially no defensible position. and Device AI is shaped by chipmakers and device OEMs, with telcos reduced to a connectivity role. That leaves the operator edge and the network edge as the only layers where a telco's physical footprint, spectrum control, and regulatory position might create real differentiation.
This three-part logic is simple, and it is politically useful inside large organisations for a specific reason. it gives every function a comfortable answer.
It gives CTOs a technical rationale for selective deployment rather than a blank check for GPU purchases.
It gives CFOs a reason to stay cautious on capital expenditure while still appearing forward-looking.
It gives strategy and corporate development teams a partnership narrative that avoids the politically awkward admission that hyperscalers will likely win most of the AI compute market regardless of what telcos do.
But the argument is thinner than it first appears, and it becomes noticeably thinner once you look at what happened the last time telcos tried something structurally similar. A few years ago, major operators made big bets on "network-edge computing" that didn't pay off. They built capacity before they had the workloads to fill it.
Early network-edge computing bets by AT&T and Cox Communications struggled to demonstrate clear return on investment and were quietly scaled back after the initial enthusiasm faded. KDDI and Bell Canada had not meaningfully expanded their edge site capacity since roughly 2022, and only revived interest in doing so once AI inference gave the concept a second narrative life. That history matters because it means the current wave of edge AI enthusiasm is, in a real sense, a second attempt at a strategy that already underperformed once on a different workload set.
The financial scrutiny is much higher today. boards will not fund empty edge data centers waiting for a hypothetical "killer app" to arrive.
2. Why Workload Placement is Everything
We are told repeatedly that we must move AI to the edge to make it fast. However, if we look at the data, we see that for most Generative AI today, the network isn't the bottleneck. For a large share of today's generative AI workloads, particularly anything built on top of large reasoning model, the network latency between the user and the nearest compute site is a small fraction of total end-to-end response time. The dominant cost is the "reasoning time" - the time the model itself spends generating tokens or working through a reasoning chain - not the milliseconds spent traversing the network. so -
If an AI takes two full seconds to "think" and generate a response, saving 20 milliseconds by putting the compute hardware at a cell tower doesn't change the user experience. It goes entirely unnoticed.
That means the "move the AI closer to the user and latency improves" argument, which underlies a large share of the network-edge investment case, is simply less true for many of today's most visible AI applications than it was for, say, real-time video conferencing or multiplayer gaming a decade ago.
The latency case for edge AI is absolutely real, but only for a specific and fairly narrow set of workloads, not for AI in general.
Most strategy reports do not draw that line clearly enough. Therefore, we should focus relentlessly on Workload Placement. We need to stop asking "should we build the edge?" and start asking "which specific AI tasks justify the high cost of local infrastructure?"
None of this means the standard narrative is wrong to focus on operator and network edge. It means the case for that focus needs to be workload-specific and evidence-based rather than asserted as a general truth about "where AI is going."
3. The Edge is a Hierarchy, Not a Place - The New AI Architecture
The most significant shift in thinking involves moving away from "The Edge" as a single, mystical location.
The most useful technical move available right now is abandoning "the edge" as a single undifferentiated place and instead treating it as a real hierarchy of infrastructure tiers, each with distinct latency, cost, and capacity characteristics.
To build a functioning AI strategy, we must focus on these three distinct layers:
1. Regional Data Centers: The Engine Room
These are the most economical locations for heavy compute. They are large-scale facilities where space is abundant and cooling systems are industrial-grade. This is where the most massive "reasoning" models should live. They can host larger models, larger batches, and more expensive accelerator hardware because power, cooling, and physical space are far less constrained than at a cell site.
If an AI task can tolerate a half-second delay, it belongs here.
From a financial perspective, this is the only place where you can run massive, power-hungry GPU clusters at a reasonable cost per unit of compute. The economics of scale still apply, and forcing heavy compute out of these centers without a strict latency requirement is an architectural error.
2. Metro and CDN-Style Nodes:
The Local Hubs Sitting closer to the end-user, these nodes are located in major cities and neighborhood hubs. They offer a genuine improvement in latency compared to a centralized cloud, but they come with a "real estate tax."
Space and power are more constrained here than in a regional center.
This is the "sweet spot" for interactive AI, things like voice assistants, real-time translation, or augmented-reality overlays etc where a slight delay feels unnatural to a human user. At this tier, you balance the need for responsiveness with the ability to pool resources across thousands of users, achieving a middle ground in unit economics.
