Skip to content
RichseenAtlas← JourneySign in

What AI Changes

How these connect

Benchmark saturation and what it relates to. Every line is an evidence-backed relation, and the arrangement is fixed — this address draws this picture, today and next year.

comparable_to — In both cases the published number moved sharply because the instrument changed rather than because the underlying subject did, and in both cases the change is documented by the body that made it.assesses — The AI Index, published by the Stanford institute Li co-directs, is the compilation that documents both the rate of benchmark movement and the growing doubt about benchmark reliability.part_of — A saturating benchmark is one specific way an evaluation stops carrying information; it is a component of the wider measurement problem rather than the whole of it.Fei-Fei LiFei-Fei LiBenchmark saturationBenchmark saturationThe difficulty of evaluating machine learning systemsThe difficulty of evaluat…Measured adoption of AI by firmsMeasured adoption of AI b…
4 drawn · 7 reachable and not drawn · positions derived from object ids, never from a simulation
Open more3
Every relation drawn, in words3
  1. Measured adoption of AI by firms · comparable_to · Benchmark saturation

    In both cases the published number moved sharply because the instrument changed rather than because the underlying subject did, and in both cases the change is documented by the body that made it.

  2. Fei-Fei Li · assesses · Benchmark saturation

    The AI Index, published by the Stanford institute Li co-directs, is the compilation that documents both the rate of benchmark movement and the growing doubt about benchmark reliability.

  3. Benchmark saturation · part_of · The difficulty of evaluating machine learning systems

    A saturating benchmark is one specific way an evaluation stops carrying information; it is a component of the wider measurement problem rather than the whole of it.