Artificial Intelligence · Deep Learning
A topic within Deep Learning, itself one of 14 topics in that field and part of Artificial Intelligence.
2 works
Kaplan et al.
Shows language model loss follows smooth power laws in model size, dataset size, and compute across seven orders of magnitude, and that within a fixed compute budget it is better to train large models on comparatively modest data and stop before convergence.
30 pageslink checked 17 Sept 2026Hoffmann et al.
Retrains the earlier Kaplan scaling laws with a wider sweep of model and data sizes and finds most large language models of the time were significantly undertrained relative to their size — model size and training tokens should scale together, roughly equally, as compute grows.
36 pageslink checked 17 Sept 2026