Image segmentation with deep methods and Theano

I am looking to evaluate some of the deep methods on a image segmentation problem familiar to the team. I am very familiar with regular MLPs but not so with image processing.

The problem is very simple, given a set of k x k images (k ~= 500), say for each pixel if it is class A or class B (e.g., sky & not sky). I have 1000's of unlabeled images and a handful labeled ones. If need be I can generate more labeled ones (with a different method) but these will not be perfect. The resolution of the segmented image should be the same as the original. I can downsample the images if need be to save computation time, but 100x100 would probably be the extreme limit.

So the goal is not unlike that in this paper by Alvarez/LeCun et. al.

I am wrapping my head around Theano and its algorithms to see how this would work but not quite there yet. I was planning to start from the LeNet example in the tutorial.

My current plan:

1. Take a sliding window (say 32x32) over an image, generating a series of overlapping patches for each image (and associated ground truth). Question: I guess the overlap depends on the application, is there a heuristic?

2. Train the net in a supervised way on every patch of every labeled image. Thus the network will have 32*32 outputs (a logistic regression layer) and 32*32 inputs each denoting a grayscale value. Question: how do people usually deal with color, does that mean 32*32*3 inputs? (for RGB)

3. Apply the trained net to a new image & stitch all the patches together (averaging on overlap) to get the final result. Probably need to do thresholding of some sort as post processing to get sharp delineations.

So I have only a handful of high quality labeled images but can generate an arbitrary number of noisy ones (should be reasonably good but not perfect).

1. How to deal with this? Should I do an unsupervised training first, followed by supervised fine tuning (on the noisy ones, good ones, both?). But then for unsupervised training I guess I need to switch to a different network type (e.g., stacked RBM / CNN hybrid or something?).

2. Could you also solve this problem with a stacked autoencoder which maps the full image onto the segmented one? Or would require too many inputs/output and/or lose context?

Any guidance appreciated.