====== Recurrent Neural Network (RNN) ======

A network architecture that processes sequences by maintaining a hidden state updated at each time step. [[papers:attention_residuals|Attention Residuals]] has a recurrent interpretation: the softmax attention over all prior layer outputs can be viewed as a weighted recurrence, connecting transformer depth to recurrent computation.

See also: [[papers:attention_residuals]], [[concepts:softmax_attention]], [[concepts:gradient_highway]]