Learning by acting, with delayed and sparse feedback.
11 topics · 19 curated works
No prior grounding assumed.
RL Course by David Silver — Lecture 2: Markov Decision Processes
David Silver (Google DeepMind) · 2015
Builds Markov decision processes up from Markov chains and reward processes, defining the value function and Bellman equation that the rest of the…
+1 more at this level
Assumes you know the vocabulary.
Apprenticeship Learning via Inverse Reinforcement Learning
Abbeel & Ng · 2004
Sidesteps recovering the exact reward function by matching the feature expectations of an expert's demonstrations, guaranteeing the learned policy…
+6 more at this level
Primary sources and full treatments.
Q-learning
Christopher Watkins & Peter Dayan · 1992
Proves that an agent can converge on optimal action values from raw experience alone, without ever building a model of the environment's dynamics.
+9 more at this level
12 of 19 works
David Silver (Google DeepMind)
Builds Markov decision processes up from Markov chains and reward processes, defining the value function and Bellman equation that the rest of the course's algorithms all solve.
link checked 17 Sept 2026David Silver (Google DeepMind)
Derives policy iteration and value iteration from the Bellman equations, treating dynamic programming as the exact solution method every approximate reinforcement learning algorithm is trying to shortcut.
link checked 17 Sept 2026Abbeel & Ng
Sidesteps recovering the exact reward function by matching the feature expectations of an expert's demonstrations, guaranteeing the learned policy performs at least as well as the expert under the true, unknown reward.
link checked 17 Sept 2026Csaba Szepesvári
A short formal survey of the main reinforcement-learning algorithms and, more usefully, of what has actually been proven about when they converge.
link checked 17 Sept 2026Mnih et al.
Shows that running many actor-learners asynchronously on CPU cores stabilises deep reinforcement learning as well as a replay buffer does, while training faster and off GPU.
link checked 17 Sept 2026Schulman et al.
Replaces trust-region policy optimisation's constrained second-order update with a clipped surrogate objective that a handful of stochastic gradient steps can optimise, trading some theoretical guarantee for practical simplicity.
link checked 17 Sept 2026Lowe et al.
Gives each agent a critic that sees every agent's actions during training but a policy that only sees its own observations at test time, which stabilises learning in environments that are neither purely cooperative nor purely adversarial.
link checked 17 Sept 2026Stiennon et al.
Applies reward modelling and reinforcement learning from human comparisons to summarisation, and finds optimising directly for human preference beats optimising for the ROUGE metric summarisation had been judged on for years.
link checked 17 Sept 2026Ouyang et al.
Fine-tunes a language model with reinforcement learning against a reward model trained on human preference comparisons, and finds human labellers prefer its smaller model's outputs to a much larger model prompted without this step.
link checked 17 Sept 2026Christopher Watkins & Peter Dayan
Proves that an agent can converge on optimal action values from raw experience alone, without ever building a model of the environment's dynamics.
link checked 17 Sept 2026Michael L. Littman
Extends Markov decision processes to Markov games, and adapts Q-learning to a minimax version so that an agent learns a policy that is robust to an adversarial opponent rather than a fixed environment.
link checked 17 Sept 2026Richard Sutton & Andrew Barto
The standard text: builds the whole field from the bandit problem up to function approximation and policy gradients.
link checked 17 Sept 2026