|
What is the difference between Recursive neural network and Recurrent neural networks. I believe that the former is used mainly for problems with temporal dimension, while the latter for Natural Language Processing (NLP). For the recursive, there are limited sources of information, and for me they are enigmatic. I would like to implement both of them, thus it would be helpful if someone could explain both concepts pithily :) Thanks in advance!! |
|
There is no difference between general recurrent neural nets and recursive nets. If a recurrent net is any neural net with recurrent connections, then recursive neural nets can be viewed as a type of recurrent neural net. However, just like when someone says "CRF" they might mean a chain connected CRF or even a linear chain CRF, when someone says "recurrent net" they might mean a particular type of computation graph that has a sequence of inputs and hidden states where the hidden state at time t is a function of the hidden state at time t-1 and the input at time t. If you truly understand backpropagation you should be able to implement a completely general recurrent/recursive neural network. That said, making assumptions that restrict what sort of connections exist in a general recurrent neural net can often yield a more efficient implementation. 1
Are we talking about this kind of recursive neural network (http://nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf) ? If so, I fail to see how to reproduce similar structure with a recursive neural network.
(Aug 23 '13 at 19:14)
Laurent Dinh
1
Recurrent neural network can be seen as a instantiation of recursive neural networks. See the question asked at end of http://techtalks.tv/talks/deep-learning-for-nlp-without-magic-part-2/58415/
(Aug 23 '13 at 21:44)
Arun Kumar
@Arun Kumar : Ok, i made a typo : replace "recursive" with "recurrent" in "I fail to see how to reproduce similar structure with a recursive neural network". gdahl seems to say that recursive net is a type of recurrent net whereas you seem to say that recurrent net is a type of recursive net. (I cannot access the URL yet)
(Aug 23 '13 at 22:53)
Laurent Dinh
Yes Laurent Dinh, exactly the algorithm im referring to is in this paper
(Aug 24 '13 at 02:54)
Issam Laradji
1
Actually I said a recursive neural net is a term for a "general recurrent neural net." I use the terms interchangeably. People like Richard Socher prefer to call recurrent neural nets with tree structure that don't have chain structure recursive neural nets. But this is all a distraction since backprop works the same on both. I personally don't like saying that tree structures are recursive and chain structures aren't since feeding in the previous hidden state as input to the function computing the next hidden state is also a recursive operation.
(Aug 24 '13 at 12:36)
gdahl ♦
|
