Models that produce samples rather than labels.
10 topics · 16 curated works
No prior grounding assumed.
MIT 6.S191 (2023): Deep Generative Modeling
Alexander Amini (MIT 6.S191) · 2023
Introduces autoencoders, VAEs, GANs and diffusion as different answers to the same question — how to learn a model that can generate new, realistic…
+1 more at this level
Assumes you know the vocabulary.
Hierarchical Neural Story Generation
Fan, Lewis & Dauphin · 2018
Introduces top-k sampling and a hierarchical generate-the-premise-then-the-story pipeline, arguing generic beam search decoding is the wrong tool for…
+3 more at this level
Primary sources and full treatments.
Auto-Encoding Variational Bayes
Diederik P. Kingma & Max Welling · 2013
Moving the randomness outside the network makes the sampling step differentiable, so a latent-variable model can be fitted by ordinary gradient…
+9 more at this level
12 of 16 works
Alexander Amini (MIT 6.S191)
Introduces autoencoders, VAEs, GANs and diffusion as different answers to the same question — how to learn a model that can generate new, realistic data rather than just label existing data.
link checked 17 Sept 2026Andrej Karpathy
Frames a language model as a lossy compression of its training corpus plus a fine-tuning stage, then argues the security problems follow directly from that construction.
link checked 17 Sept 2026Fan, Lewis & Dauphin
Introduces top-k sampling and a hierarchical generate-the-premise-then-the-story pipeline, arguing generic beam search decoding is the wrong tool for open-ended, creative generation even when it is right for translation.
link checked 17 Sept 2026Radford et al.
Shows a large autoregressive language model trained only to predict the next word on web text begins performing translation, summarisation and question answering it was never explicitly trained for.
link checked 17 Sept 2026Holtzman et al.
Shows maximum-likelihood decoding produces repetitive, generic text precisely because it is high-probability, and proposes nucleus sampling — truncating to the smallest set of tokens covering a probability mass — as a fix.
link checked 17 Sept 2026Wei et al.
Documents capabilities that appear sharply once a language model passes a scale threshold, absent at smaller scale on the same task — a discontinuity since disputed as an artefact of how the metrics were chosen rather than a real property of the models.
link checked 17 Sept 2026Diederik P. Kingma & Max Welling
Moving the randomness outside the network makes the sampling step differentiable, so a latent-variable model can be fitted by ordinary gradient descent.
link checked 17 Sept 2026Goodfellow et al.
Trains a generator against a discriminator so that neither needs an explicit likelihood, and set the direction of generative modelling for a decade.
9 pageslink checked 17 Sept 2026Sohl-Dickstein, Weiss, Maheswaranathan & Ganguli
Destroy structure in data with a slow noising process, then learn to reverse it: the result is a generative model with a tractable likelihood, and the basis of diffusion.
link checked 17 Sept 2026Rezende & Mohamed
Shows that composing a sequence of invertible transformations turns a simple base distribution into an arbitrarily complex one with a tractable density, making a VAE's variational posterior far richer than a diagonal Gaussian.
link checked 17 Sept 2026van den Oord, Kalchbrenner & Kavukcuoglu
Models an image as a sequence of pixels generated one at a time by a recurrent network, showing that tractable exact-likelihood autoregressive models could match or beat GANs on sample quality of the time.
link checked 17 Sept 2026van den Oord et al.
Generates audio one sample at a time with a dilated causal convolutional network, producing far more natural speech than the concatenative and parametric methods it replaced at the cost of very slow generation.
link checked 17 Sept 2026