In the previous section (how to understandRNN? (Theory)) The course introduces the basic structure of the cyclic neural network. At the same time, such a loop structure will bring some difficulties to the optimization. This paper mainly introduces two simple ways to alleviate the optimization problem of RNN.
