Sequential Model

• Naive sequence model

• Autoregressive model

• Markov model (first-order autoregressive model)

• Latent autoregressive model

Recurrent Neural Network

• Short-term dependencies / long-term dependencies

• Problem of RNN

Long - Short Term Memory

• Forget Gate