Deep Learning
Neural networks, and why depth turned out to matter so much.
13 topics · 2 curated works
Topics
- 01Backpropagation
- 02Multilayer Perceptrons
- 03Convolutional Neural Networks
- 04Recurrent Networks & LSTMs
- 05Transformers & Attention
- 06Normalisation Techniques
- 07Optimisers & Learning Rates
- 08Initialisation
- 09Regularisation in Deep Nets
- 10Scaling Laws
- 11Neural Architecture Search
- 12Representation Learning
- 13Interpretability & Mechanistic Analysis
Curated reading
2Go deeper
Primary sources and full treatments.
- PaperAttention Is All You NeedVaswani et al., 2017· 15 pages· Free
Replaces recurrence with self-attention, producing the Transformer architecture that underpins essentially all current language models.
- BookDeep LearningGoodfellow, Bengio & Courville, 2016· Free
The reference textbook for the mathematical foundations underneath modern neural network practice.