Initialisation

A topic within Deep Learning, itself one of 14 topics in that field and part of Artificial Intelligence.

Reading on Initialisation

2 works

Paper2010

Understanding the Difficulty of Training Deep Feedforward Neural Networks

Glorot & Bengio

Traces the difficulty of training deep networks to how weight initialisation interacts with activation functions and layer-to-layer signal variance, and derives an initialisation scheme (Xavier initialisation) that keeps that variance stable across depth.

8 pageslink checked 17 Sept 2026

Artificial Intelligence · Deep Learning

FreeAdvanced

Paper2015

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

He, Zhang, Ren & Sun

Shows the standard Xavier initialisation assumes the wrong activation function for ReLU networks, and derives a variance-preserving initialisation for ReLUs that makes much deeper networks trainable from scratch.

11 pageslink checked 17 Sept 2026

Artificial Intelligence · Deep Learning

FreeAdvanced

Reading on Initialisation

Understanding the Difficulty of Training Deep Feedforward Neural Networks

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Other topics in Deep Learning