|
Hi, Hi, I modified the RBM given in the Deep Learning tutorials from LISA Lab. I added Gaussian units for the visible layer according to recommendations I found at the theano-users page. I applied it on real world images taken from the PASCAL dataset. I have resized the images to a reasonable size say 37*50, and converted them to grayscale. At first my training cost seems to reduce, but all of a sudden it explodes These are the parameters of my RBM: test_rbm(dataset='../../Datasets/Pascal/VOC2007/VOCdevkit/VOC2007/JPEGSmall/Gray/pascal_small.pkl.gz',learning_rate=0.001, training_epochs=430, batch_size=20, n_chains=20, n_samples=10, output_folder='rbm_pascal', n_hidden=500) A few samples from how my cost progresses: Training epoch 321, cost is -717.034 Training epoch 353, cost is -706.652 Training epoch 394, cost is -681.335 Training epoch 435, cost is -578.763 Training epoch 436, cost is -575.831 Training epoch 437, cost is -558.255 Training epoch 438, cost is -602.048 Training epoch 439, cost is -593.737 Training epoch 440, cost is -653.543 Training epoch 441, cost is -870.619 Training epoch 442, cost is -922.471 Training epoch 443, cost is -941.04 Training epoch 444, cost is -934.836 Training epoch 445, cost is -1485.02 Training epoch 446, cost is -2626.16 Training epoch 447, cost is -3284.69 Training epoch 493, cost is -137838 Training epoch 494, cost is -158208 Training epoch 518, cost is -962159 Training epoch 519, cost is -1.02229e+06 Training epoch 520, cost is -1.05705e+06 Training epoch 540, cost is -6.40528e+06 Training epoch 544, cost is -9.5827e+06 As you can see it suddenly starts exploding.Also my filters look nothing like edge,point etc detectors. Apart from not having fully converged, could this also be due to the dataset choice being too complex? I tried training on caltech 101 faces but again ended up with explosions in cost. Could anybody suggest any reason for this happening? I can't seem to understand it in terms of local minima etc. Thanks in advance |
