|
Hello I am pretty new to ML learning and even newer to NN architectures. However I have found autoencoders to be an amazing inovation in image classification hence I have decided to use them. In this direction I have downloaded a deep learning toolbox implementation in Matlab and start running some initial tests on the MNIST dataset. The code I am using can be found in this link below: https://github.com/rasmusbergpalm/DeepLearnToolbox The features I have managed to construct over the MNIST dataset are very interesting and definetely similar to the ones I have seen in many popular puplications. However the my problem arises when I am trying to move to a new dataset. More precisely I am trying to construct features from satellite images. I have tiled my initial images in various sizes (from 128x128 down to 8x8) but still I most of the features I have managed to construct are just pure noise, very different from the MNIST features. I have even decreesed my dataset from 30000 data down to 1000 but still no sucess. In some few cases I have managed to construct some non-noise features but in this cases all of them are just copies of each other so no variability at all. I have tried multiple architectures by changing the number of hiden units in my sample but still no sucess. Until know all my architextures are swallow as my autoencodes use just one hidden layer but I am not sure if I should move to deeper model using noise generated features from the fist process. So I am really stuck and I do not understand why autoencoders fail so badly over my data but seem to work perfectly over the MNIST dataset. Is there any limitation in the data that autoencoders can model ? Can someone point out any helpful link trying to deal with such a problem ? Thank you in advance
This question is marked "community wiki".
|
|
Why dont you try to take some patches from your images and try to learn the features? And moreover satellite images might need additional pre-processing. You can have a look at UFLDL tutorial (http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial) You can also see this work on predicting roads from satellite images: www.cs.toronto.edu/~hinton/absps/noisy_maps.pdf
This answer is marked "community wiki".
Thanks this paper looks interesting ! However what exactly do you mean by 'taking some patches and try to learn features'. I think this is exactly what I am doing and failing. The UFLDL tutorial seems good for someone who wants to program the thing from the begging. I already have an implementation which works very nicely with MNIST data however I can not expand this nice outcomes to new datasets !
(Feb 11 '14 at 05:11)
DimitrisM
I also faced similar issues with the Deep Learning tool box. You should try to vary the parameters and see if you can get good results. This is all about parameter tuning, I am realising - so this will ensure what features you learn. By parameters I mean Number of Hidden neurons, what type of neuron it is, wt decay, sparsity, whitening of inputs etc.
(Feb 12 '14 at 04:35)
Sharath Chandra
Thank you for the tips. However can someone point out why this seems to work relatively easy with MNIST benchmark but seems to fail whent trying to process other kinds of images ? I mean what makes so different the process of testing different images ? Do you also have any good reference taling about parameter tunning ? Thanks again for the tips.
(Feb 17 '14 at 04:08)
DimitrisM
You can make parallels for SAE also I guess.
(Feb 17 '14 at 04:40)
Sharath Chandra
Thanks again for the fast and interesting feedback. Could you also propose any paper regarding the preprocessing of data when these are higly diverse such as in my case ? Best Regards
(Feb 18 '14 at 07:46)
DimitrisM
|
