|
Hallo, I have to deal with relatively high dimensional data. My data set consists of roughly 1800 images, with each image having a size of 144 x 176 (25344 pixels). The images contain observations of an industry process that are relatively monotonic with some exceptions when something goes wrong. I want to use a neural network to learn invariant features (rotation, lightness and brightness). Has any one suggestions which kind of neural network, regularization and training method would be appropriate? I already tried to learn features using stacked auto-encoders (SAE) and deep belief networks (DBN). Further, I achieved my best results using dropout, L2 regularization, mini-batch SGD and logistic units (logistic hidden and linear visible units in case of DBN). It learns reasonable features (to distinguish classes) on MNIST or Cifar. But on my industry process data the activation in the bottleneck layer is almost identical with few exceptions. I am currently working on a convolutional network but it is highly complicated for me. My Matlab implementation of max-pooling is not fast enough to allow serious training times. Thanks. |
|
Actually I found a fast solution for Max-pooling when using matlab. This site blockproc for max-pooling how to use a Matlab image processing function for max-pooling. The function performs a Distinct block processing for an image. You can specify the image in the first argument, the block size in the second and the target function (max of block) in the third argument. The only problem would be if you need the position of the maximum inside a block for the learning algorithm. So far I don't need it. |
|
If your main problem is to learn invariant features, I bet your best option is to stick with convolutional networks (CNN). If you managed to understand and use SAE and DBN, your just need to give it more time and everything will make sense. Here are some good resources which could help you: If you have a GPU available, the Python implementations from the last two links could dramatically decrease you learning time. It's also usually a good idea to use someone's else tested and validated implementation instead of going with one's own. Good Luck! Thank you Saul! I keep going :)
(Oct 04 '13 at 14:47)
gerard
|
