◉Artificial Intelligence

The statistical learning theory underneath everything else here.

Supervised Learning · Unsupervised Learning · Bias-Variance Tradeoff · Overfitting & Regularisation · Cross-Validation

Neural networks, and why depth turned out to matter so much.

Backpropagation · Multilayer Perceptrons · Convolutional Neural Networks · Recurrent Networks & LSTMs · Transformers & Attention

Getting machines to handle language, which turned out to be most of the problem.

Tokenisation · Word & Sentence Embeddings · Language Modelling · Machine Translation · Question Answering

Extracting structure from images.

Image Classification · Object Detection · Semantic Segmentation · Pose Estimation · Optical Flow

Learning by acting, with delayed and sparse feedback.

Markov Decision Processes · Dynamic Programming Methods · Q-Learning · Policy Gradient Methods · Actor-Critic Algorithms

Models that produce samples rather than labels.

Autoencoders & VAEs · Generative Adversarial Networks · Diffusion Models · Autoregressive Models · Normalising Flows

Whether these systems do what we intend — the field's most consequential open problem.

The Alignment Problem · Specification Gaming · Reward Hacking · Interpretability for Safety · Scalable Oversight

Learning with explicit uncertainty.

Graphical Models · Bayesian Networks · Hidden Markov Models · Gaussian Processes · Variational Methods

The engineering that turns a model into a product.

Training Infrastructure · Distributed Training · Model Serving & Inference · Quantisation & Distillation · Data Pipelines

The half-century of AI that came before deep learning, much of it still useful.

Search Algorithms · Constraint Satisfaction · Knowledge Representation · Expert Systems · Automated Planning

Who is accountable when the model is wrong.

Algorithmic Bias & Fairness · Transparency & Explainability · Privacy & Data Rights · Labour Market Effects · Autonomous Weapons

Recurring hype cycles, and the questions that never resolved.

The Dartmouth Conference · The AI Winters · The Turing Test · The Chinese Room Argument · Symbolic vs Connectionist Debate