Inference Costs Are Deciding Which AI Products Survive
Training budgets get the headlines, but serving costs are quietly killing products. Six teams shared their per-request economics, and the pattern is consistent.
Maya Iyer4 min read
Senior Correspondent, AI and Infrastructure
Maya Iyer
Covers the economics of machine learning systems, from accelerator supply through to the serving costs that decide which products ship.
- Machine learning infrastructure
- Semiconductors
- Cloud economics
- Model evaluation
3 articles
Open Models Closed the Benchmark Gap. Deployment Is the Gap Now.
On published evaluations the difference has narrowed to noise. Teams running both in production describe a gap that benchmarks do not measure at all.
Maya Iyer3 min read
Advanced Packaging, Not Lithography, Is the Real Chip Bottleneck
Everyone watches EUV tool shipments. The constraint on accelerator supply has moved downstream to packaging capacity, and it does not scale on the same timeline.
Maya Iyer3 min read