Training compute

Assembled from 22 blocks · 1 evidence · 26 related

1. Story
2. The total arithmetic performed to train a model, conventionally reported in floating-point operations, and the field's only widely comparable input measure. It is an estimate rather than a reading: for most models it is reconstructed from hardware counts, run durations and utilisation assumptions, because developers rarely publish it. Its usefulness is that it is the quantity everything physical attaches to — accelerator supply, capital expenditure, electricity, land and cooling water all scale with it — and its danger is that it is easy to treat as a proxy for capability when the mapping between them has never been stable. Regulators have nonetheless adopted compute thresholds, because a number that can be estimated from procurement is administrable in a way that a capability claim is not. That is a decision about tractability, not a finding about what the number means.
3. Knowledge
4. Growth in frontier training compute
5. Training compute
6. The AI accelerator
7. Connections
8. The AI accelerator
9. Growth in frontier training compute
10. Deep learning
11. The transformer architecture
12. Training compute
13. Taiwan
14. A deep convolutional network wins the ImageNet challenge
15. The transformer architecture
16. Training compute
17. Data-centre electricity demand
18. Northern Virginia data-centre corridor
19. Epoch AI
20. The IEA publishes its first global analysis of energy and AI
21. Evidence
22. Training compute for frontier AI models grew by roughly 4–5× per year from 2010 to 2024, with frontier language models growing at about 5× per year since 2020; Epoch also maintains a database of large AI data centres built from satellite imagery and permit filings, with concentrations of covered sites in Texas, Virginia, Ohio, Nebraska and Iowa. V55 verification basis: epoch.ai was not fetched. The 4–5× and 5× growth rates were carried in search results attributing them to Epoch's own published trends analysis of that title, and the data-centre database and its state-level site concentrations were carried with attribution to Epoch. Forward projections of compute growth, coverage percentages for the data-centre database, and GPU-equivalent counts appearing in secondary summaries were NOT confirmed and are excluded from this pack.