GLUE is saturated and SuperGLUE is built to replace it
node
The GLUE language-understanding suite was released in 2018 and within about a year pretrained transformer models had passed its non-expert human baseline. SuperGLUE was constructed in 2019 explicitly as a harder successor, opening with a gap of roughly eighteen points between the best available model and the human baseline — and closed similarly quickly. The two-year cycle established here has repeated at every subsequent level of difficulty and is the origin of the pattern the Journey later examines: an instrument built to discriminate, answered, and replaced faster than the interpretation of its scores could be established.
A node is not a place. Drawing it on a map would assert something about the world that no stored fact supports.
Read in · 1
Timeline
No dated observations are stored for this object. Atlas shows what was observed and when — it does not infer a history.
Connections · 2
- Benchmark saturationconcerns
- The transformer architectureconcerns
Assembled narrative · 1
An assembled narrative is available for this object.
Observed changes · 0
No public Signals are attached to this object. Signals show what changed and when it was observed — never a direction or a rank.
Actions
/atlas?object=EVT_GLUE_SUPERGLUE_2018_2019