Particle Swarm Optimization in Deep Learning for Image Recognition

There are definitely some papers about using SI and genetic algorithms (GA) for training NN (just search in google scholar). But after reading some papers I got a feeling that you should use these kind of algorithms in a different way rather than for directly training NN.

To get a general idea I recommend you to read G. Hinton's paper Where Do Features Come From? Quote from the paper:

"Evolution is much too slow to discover the millions of features we need. In very high-dimensional spaces, searches that have efficient access to gradient information are millions of times faster than searches that do not ( It is always possible to use a local random search to estimate the gradient but in spaces with millions of dimensions, this is millions of times slower than methods like backpropagation that compute the gradient efficiently). Evolution can optimize hundreds or even thousands of parameters, but it is hopelessly inefficient for optimizing millions of parameters because it cannot compute the gradient of the fitness of the phenotype with respect to heritable parameters. What evolution can do is explore the space of biological devices that can make effective use of gradient information. It can also explore the space of objective functions that these devices should optimize and the space of architectures in which this optimization works well."

Y. Bengio have also done some analysis and experiments in this direction, take a look at this paper Evolving Culture vs Local Minima

So the general filling that I got is direct training parameters of the deep NN with SI or GA is not that you should really do with these algorithms.