User Tools

Site Tools


concepts:gradient_highway

Gradient Highway

The property of residual connections that allows gradients to flow directly through the identity path during backpropagation, bypassing layer transformations. This enables stable training of very deep networks.

See also: residual_connections, vanishing_gradients, attention_residuals

concepts/gradient_highway.txt · Last modified: by aethersync

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki