Growth in frontier training compute

node

The most reliably measured trend in the field, and the one most often mistaken for a law. Epoch AI, which maintains a public database of notable models and reconstructs the arithmetic operations used to train them, reports that training compute for frontier models grew by roughly four to five times a year from 2010 to 2024, with frontier language models growing at about five times a year since 2020. That is far faster than semiconductor performance improved over the same period, so most of the increase is not from better chips: it is from spending more, on more accelerators, for longer. Two things follow that are frequently conflated. The trend is a measurement of inputs, not of capability, and the relationship between the two is empirical and imperfectly characterised. And an exponential in spending is bounded by things exponentials in physics are not — capital, electricity supply, grid connections and advanced-node fabrication capacity — which is why the interesting question about the trend is not how long the mathematics permits it to continue but which of those constraints binds first.

A node is not a place. Drawing it on a map would assert something about the world that no stored fact supports.