Are there any painless method to implement LSTM training?

It is really hard to implement the gradients manually for LSTM.

So I'm looking for either: 1) a paper with clear derivative formulas 2) a ready to use package to ease the pain of gradients handcrafting

I have tried Theano but it takes surprisingly long to precompute gradients and I found no tricks to cache the result of the gradients computation.

Thanks!

asked Dec 20 '13 at 03:58

Konstantin
341●8●12●18

2 Answers:

I don't see what takes long with the LSTM gradients. I used it quite sucessfully a while back.

That being said, the Phd thesis of Alex Graves has them all, I think. Also, PyBrain has code here:

answered Dec 23 '13 at 15:59

Justin Bayer
1706●9●30●45

It should be noted that Alex Grave's thesis uses BPTT rather than the original LSTM algorithm that's based on RTRL.

(Dec 28 '13 at 15:26) Max

1	You can find an tutorial how to implement lstm with theano: here answered Dec 18 '14 at 13:29 Christian H 16●1

Once you sign in you will be able to subscribe for any updates here

Tags:

theano ×28
rnn ×13
lstm ×1

Asked: Dec 20 '13 at 03:58

Seen: 2,209 times

Last updated: Dec 18 '14 at 13:29