Strange behavior when implementing Back propagation in DBN

Hi,

Currently I'm trying to implement the Deep Belief Network. But I've met a very strange problem. My source code can be found here: https://github.com/mistree/GoDeep/blob/master/GoDeep/

I first implemented the RBM using CD and it works perfectly (by using the concurrency feature of Golang it's quite fast). Then I start to implement a normal feed forward network with back propagation and then the strange thing happens. It seems very unstable. When I run it with xor gate test it sometimes fails, only when I set the hidden layer nodes to 10 or more then it never fails. Below is how I calculate it

Step 1 : calculate all the activation with bias

Step 2 : calculate the output error

Step 3 : back propagate the error to each node

Step 4 : calculate the delta weight and bias for each node with momentum

Step 1 to Step 4 I do a full batch and sum up these delta weight and bias

Step 5 : apply the averaged delta weight and bias

I followed the tutorial here http://ufldl.stanford.edu/wiki/index.php/Backpropagation_Algorithm

And normally it works if I give it more hidden layer nodes. My test code is here https://github.com/mistree/GoDeep/blob/master/Test.go

So I think it should work and start to implement the DBN by combining the RBM and normal NN. However then the result becomes really bad. It even can't learn a xor gate in 1000 iteration. And sometimes goes totally wrong. I tried to debug with that so after the PreTrain of DBN I do a reconstruction. Most times the reconstruction looks good but the back propagation even fails when the preTrain result is perfect.

I really don't know what's wrong with the back propagation. I must misunderstood the algorithm or made some big mistakes in the implementation.

If possible please run the test code and you'll see how weird it is. The code it self is quite readable if you ignore the "go func()" statements. Any hint will be great help.Thanks in advance