Artificial Intelligence · Machine Learning Systems
A topic within Machine Learning Systems, itself one of 11 topics in that field and part of Artificial Intelligence.
2 works
Geoffrey Hinton, Oriol Vinyals & Jeff Dean
Trains a small model to match the soft output probabilities of a large trained model rather than its hard labels, showing that the large model's mistakes carry information a small model can learn from directly.
link checked 17 Sept 2026Benoit Jacob et al.
Works out how to train a network so both its weights and activations can run entirely in 8-bit integer arithmetic at inference, closing most of the accuracy gap that naive post-training quantisation leaves open.
link checked 17 Sept 2026