If you’re pushing serious AI workloads, there’s a point where buying your own GPUs quietly beats “just use the cloud” on pure maths. Once you cross that point, the savings stop being theoretical and start showing up in your P&L.
This is 4th episode in the series of Build vs Buy vs Rent: The AI Infrastructure Decision Tree for Startups.
TL;DR
Above ~70% GPU utilisation, owning hardware usually beats the cloud on cost.
The real break‑even sits roughly between 55–75% utilisation, depending on power, amortisation, and cloud pricing.
Lenovo’s TCO work: ~8x cheaper than cloud infra and up to ~18x cheaper than frontier APIs per million tokens.
At 10B tokens/month, three‑year savings vs pure APIs can exceed £2M.
8x H100 server: $250k–$400k upfront plus $3k–$5k/month to run; ~$11k/month effective cost over three years.
Equivalent cloud H100 capacity: roughly $14k–$20k/month.
You must factor in power (6–10 kW per rack unit), UK colocation (£500–£2,000/rack/month), and infra engineers (£80k–£140k/year).
A hybrid model (own the baseline, rent the spikes) can cut AI infra spend by ~40–60% vs pure cloud.
At around 70 percent GPU utilisation, owning on‑prem infrastructure is usually the better deal for AI inference. Below that level, the cloud’s flexibility earns its premium because you’re not paying for expensive hardware that sits idle when traffic drops. In practice, the break‑even lives somewhere between 55 and 75 percent sustained utilisation, depending on things like your electricity rate, how many years you plan to amortise the kit, and which cloud pricing tier you’re comparing against.
Lenovo’s 2026 total cost of ownership work puts real numbers to this. They found that self‑hosted GPU infrastructure can be about 8 times cheaper per million tokens than raw cloud infrastructure, and up to 18 times cheaper than using frontier model APIs. Once you’re at around ten billion tokens a month, the three‑year cost gap between owning hardware and living entirely on cloud APIs can easily exceed two million pounds.
The sticker price on an 8‑GPU H100 box is not small. You’re looking at roughly 250,000 to 400,000 dollars upfront for the server itself. Then you add 3,000 to 5,000 dollars a month in operating costs for colocation, power, cooling, and maintenance.
Spread the hardware over three years and your effective monthly cost lands at around 11,000 dollars.
Buying similar H100 capacity on‑demand in the cloud typically ends up between 14,000 and 20,000 dollars a month, so the saving is real and it compounds over time.
Where founders often get caught out is in the hidden line items.
Modern H100 servers can pull 6 to 10 kilowatts per rack unit, which means you can’t just drop them into a normal office and hope for the best.
You need proper co-location, and in the UK that runs about 500 to 2,000 pounds per rack per month. You also need engineers who can run GPU infrastructure safely and reliably, and UK market rates put that at roughly 80,000 to 140,000 pounds per person per year. On top of that, you’re carrying hardware risk:
once you buy, your performance ceiling is locked in for the amortisation period while cloud alternatives quietly keep improving in the background.
That’s why most serious AI teams don’t go “all cloud” or “all on‑prem” for long. The model they converge on is hybrid: own enough hardware to cover your predictable baseline workloads, and use the cloud when you need to absorb burst traffic.
if planned well, that blended architecture cuts your overall AI infrastructure bill by about 40 to 60 percent compared to living entirely in the cloud.
It’s where most AI companies end up once their volume of inference forces them to care about infrastructure as more than a line item.
Listen to the full episode here, in Substack app, or Apple, Spotify / youtube.
This week Intelligent Founder AI and AI Unfiltered broke into Substack’s “Rising in Technology” leaderboard at #88—together, after already hitting #98 in Business earlier this year. Thank you for all the support so far.
AI Unfiltered is where we track the latest AI news, investigations, and “what just happened?” moments and, more importantly, what they actually mean for operators, founders, and buyers. Think model‑on‑model training fights, national‑security angles, regulatory shifts, and how all of that moves the ground under your product roadmap.
Intelligent Founder AI goes deeper: long‑form dives into technical and business architecture, operator‑grade GTM, and governance playbooks for getting AI into real enterprises and critical infrastructure without getting lost in the hype. It’s where we turn those headlines into concrete frameworks you can actually run inside your own stack.
Together, AIU and IF.ai are designed as a pair:
one keeps you on top of what’s changing week to week, the other helps you re‑wire your architecture, strategy, and sales motion around it. If that sounds useful, check out both, hit subscribe, and tell us what you want us to dissect next.
