Artificial Intelligence · Generative Models
A topic within Generative Models, itself one of 10 topics in that field and part of Artificial Intelligence.
No prior grounding assumed.
Intro to Large Language Models
Andrej Karpathy · 2023
Frames a language model as a lossy compression of its training corpus plus a fine-tuning stage, then argues the security problems follow directly…
Assumes you know the vocabulary.
Language Models are Unsupervised Multitask Learners
Radford et al. · 2019
Shows a large autoregressive language model trained only to predict the next word on web text begins performing translation, summarisation and…
+1 more at this level
Primary sources and full treatments.
Are Emergent Abilities of Large Language Models a Mirage?
Schaeffer, Miller & Koyejo · 2023
Argues apparent emergent abilities mostly disappear when a task is scored with a smooth, per-token metric instead of a nonlinear all-or-nothing one,…
4 works
Andrej Karpathy
Frames a language model as a lossy compression of its training corpus plus a fine-tuning stage, then argues the security problems follow directly from that construction.
link checked 17 Sept 2026Radford et al.
Shows a large autoregressive language model trained only to predict the next word on web text begins performing translation, summarisation and question answering it was never explicitly trained for.
link checked 17 Sept 2026Wei et al.
Documents capabilities that appear sharply once a language model passes a scale threshold, absent at smaller scale on the same task — a discontinuity since disputed as an artefact of how the metrics were chosen rather than a real property of the models.
link checked 17 Sept 2026Schaeffer, Miller & Koyejo
Argues apparent emergent abilities mostly disappear when a task is scored with a smooth, per-token metric instead of a nonlinear all-or-nothing one, making the discontinuity a property of the chosen metric rather than the model.
link checked 17 Sept 2026