The algorithm for computing gradients of a loss function with respect to network parameters by applying the chain rule layer by layer from output to input. Vanishing gradients occur when these signals decay excessively in deep networks. Residual connections create the gradient highway that preserves gradient flow.

See also: vanishing_gradients, gradient_highway, residual_connections