How Many Will Actually Get Built & Is Energy AI's BIGGEST Bottleneck? | Positron AI Co-founder

Why does it matter? Because the companies everyone says are burning cash are already printing it

Thomas Sohmers co-founded Positron AI — $875M Series C, $5B valuation — and builds the hardware that serves every frontier model. What he reveals about AI lab economics, inference architecture, and geopolitics should revise assumptions most investors and developers are still carrying.

• Anthropic's API business reportedly runs at ~80% gross margin right now — not eventually.
• Processing a cached token costs providers 1/1000th of generating a new one; most charge nearly full price for it.
• Anti-data center politics uniting American left and right is, in Thomas's view, almost entirely a Chinese influence operation.
• GPT-6 Astra compressed a 2–3 week chip design task into roughly 50 hours, navigating legacy EDA tools with almost no documentation in its training data.

OpenAI and Anthropic could become massively profitable overnight — they just choose not to

The burning-cash narrative is the most persistent myth in AI investing. Thomas is direct about it: "If they stopped training, they'd be massively profitable overnight." Every dollar of deficit is a training budget decision, not a structural business problem. Strip out that discretionary spend and what remains is a high-margin software business at scale.

Anthropic's API business is reported to carry roughly 80 points of gross margin. Thomas's reaction to that number was not surprise at the margin — it was surprise that anyone still doubts it. "It's absurd to me that the meme of OpenAI, Anthropic, etc. are just burning cash and eventually they'll run out of cash that they can burn."

The implication: stop treating frontier AI labs as speculative moonshots dependent on external funding. They are high-margin software companies choosing to reinvest aggressively in training. The moment they stop, the economics flip. That is a strategic choice, not a structural constraint — and misreading the distinction leads to badly wrong conclusions about the sector's durability.

Cached tokens cost 1/1000th of a new token to serve — and providers charge you almost full price

"Processing a cached token is essentially free. It's 1/1000th of the cost." That is the mechanical explanation for why Anthropic runs ~80% gross margins on its API. When your prompt reuses tokens the provider has already computed — a long system prompt, a persistent document, a multi-turn agentic context — the provider's marginal cost collapses. Their published cache-read discount does not reflect that collapse.

"They're making obscene margin on that cache read," Thomas says. "You make all of your money on selling cached input and output tokens."

Developers evaluating AI provider pricing typically compare headline per-token rates. Almost none of them model actual cache hit rates for their workload. For agentic coding sessions — the fastest-growing AI use case — real traced data shows roughly 96% of all tokens processed are cached. That means the margin gap between published pricing and actual provider economics is not a rounding error. It is the business.

The American left-right coalition against data centers is, Thomas argues, running on fabricated facts seeded by China

A single In-N-Out uses more water than the largest data centers in the United States. Thomas is not advancing that as a talking point — he is correcting claims that have become politically mainstream. "The scariest thing to me on the political spectrum is that it's now become almost a unifying issue on left and right about being anti-data centers. And I think that is almost entirely a Chinese scop."

The asymmetry is stark. China is adding gigawatts of new generation capacity, bulldozing homes, and running rolling blackouts to expand training infrastructure. The US has Republican governors backing away from pro-growth positions because their constituents have absorbed false claims about electricity costs and water usage — claims that happen to benefit the one country not subject to democratic opposition.

Thomas is not anti-environmental review. He wants data centers to be as architecturally impressive as the Great Pyramids. But he is clear: the specific factual claims powering current opposition do not hold up, and their coordinated spread across ideological lines is not accidental.

Local LLMs are a multiplier for cloud token demand, not a substitute

80–85% of all tokens consumed today flow through the top four model companies. The threat that on-device or enterprise-hosted models cannibalize that is, Thomas argues, backwards causality.

Right now, the binding constraint on cloud AI consumption is a human deciding when to ask something. A local model on your phone doesn't wait. "If they have a local LLM that is constantly checking their email, their calendar, messages, etc. and deciding to do lookups to cloud-hosted models frequently, that's now on a per-person basis a massive increase in the number of tokens being consumed by cloud models."

"The next really big order of magnitude increase in token volumes is going to come when us humans trust a local LLM" to initiate routing autonomously. The misconception — that every query handled locally is one fewer query for OpenAI or Anthropic — ignores that local models remove humans as the bottleneck entirely. Volume goes up, not down.

GPU compute improved 120x in a decade — memory bandwidth only 17x, and that gap is the entire inference problem

Between 2014 and 2024, a single Nvidia GPU improved 120-fold on raw FLOPS. Memory bandwidth over the same period improved 17x. That divergence is not a footnote — it is why inference requires fundamentally different hardware than training.

Training is compute-bound: parallelize across the corpus, throw more FLOPS, get a better model. Inference is memory-bound: for every single generated token, the system must read all model weights from memory. Autoregressive generation cannot be parallelized the way training batches can. "That forward pass, that inference portion of it, is heavily memory-bound," Thomas explains.

The SRAM cell — the core on-chip memory primitive — has not meaningfully scaled with Moore's Law in roughly 15 years. Architectural innovation kept compounding on the compute side. The memory side lacked the same pressure until Transformer models made bandwidth the dominant bottleneck. Companies solving memory capacity and bandwidth are solving the right problem for inference. Raw FLOP races are optimizing for training.

96% of tokens in real agentic coding sessions are cached — and most inference hardware is benchmarked against the other 4%

Semi Analysis traced actual Claude Code agentic sessions — dozens to hundreds of turns, sub-agents, multi-file operations — and found roughly 96% of all tokens processed were cached. Not a synthetic benchmark. Production workloads.

At long context lengths, individual user KV caches run to 100 gigabytes. With 50 concurrent users, their combined session context exceeds the size of a model like GPT-4's weights. Managing the tiered cache hierarchy — accelerator memory, host DRAM, NVMe, network-attached storage — becomes the core engineering challenge. "If you know your workload is going to have this extremely high caching rate, that drastically shifts the importance of how you can retrieve those caches."

Persistent KV caches save operators money more than they improve user latency. "It's mostly an economics thing for the service provider." Inference hardware benchmarked against single-turn generic queries is being evaluated against the wrong workload for the fastest-growing segment of AI usage.

GPT-6 Astra completed a full chip design in 50 hours through tooling it had almost no training data on

Electronic design automation tools were built in the 1990s. Unintuitive, barely documented publicly, largely absent from any web corpus. Models trained on internet data have essentially no innate knowledge of them. Thomas has tested every major model release on the same task: take an encryption block from RTL specification all the way through to a tape-out-ready chip file using TSMC PDKs.

GPT 5.6, six weeks before this episode, could not complete it. GPT-6 Astra did — in a little over 50 hours, timing above a gigahertz. A competent engineer doing this for the first time takes two to three weeks. "Compressing that 2 to 3 weeks down to 2 days and change — it's still mindboggling."

Thomas calls GPT-6 Astra AGI, and means it specifically: the ability to navigate complex multi-step technical processes through legacy tooling with almost no relevant training data is a qualitative leap that aggregate benchmark scores do not capture.

Token prices fell 60x — but value per token rose 100 to 1,000x, making cost-per-token a misleading unit

The Silicon Data token price index dropped below $1 per million tokens this month. Five years ago it was $60. That looks like commodity collapse. Thomas thinks framing it that way is analytically wrong.

"A $60 token 5 years ago — no one would pay a cent for today." GPT-3 was the best model in the world from August 2020 through November 2022. Run it now against any production task. The capability of the unit changed as much as the price, and in the opposite direction.

"The value per token in terms of what it can improve a person's life is orders of magnitude — I would say 100 or a thousandfold." The 60x price decline therefore represents something closer to a 1,000x improvement in value-adjusted terms. ROI frameworks that normalize purely on token cost are systematically underestimating what AI investment actually delivers — and that error compounds as capabilities keep improving faster than prices fall.

The real bottleneck was never hardware — it was human trust

Every constraint Thomas identifies converges on the same chokepoint: humans deciding when to let AI act. Local models remove that friction by monitoring and routing without waiting for a prompt. Once that trust threshold tips, the token volume implications are not incremental — they are orders-of-magnitude growth against infrastructure that was sized for a world where humans remain in the loop.

The hardware race, the margin story, the geopolitical contest over data centers: all of it is prologue to the moment AI systems are trusted to initiate rather than respond. That moment is arriving faster than the infrastructure is ready for it.

---

Topics: AI inference, semiconductor hardware, KV caching, token economics, data centers, AI profitability, memory wall, scaling laws, GPT-6, Positron AI, energy policy, on-device AI, agentic AI, chip design, Chinese AI competition