Theano + PyBrain, it is possible to combine them?

Question

Is there a way to combine Theano with PyBrain? I would like to be able to express my models in Theano (and let it compute the corresponding 1st and 2nd derivatives for me) but exploit the optimization algorithms in PyBrain (such as the Stochastic Search)?

To put it in different way: are all the optimization methods in PyBrain bound to the neural networks representation or is there a way to apply them to any python defined cost function with supplied derivatives?

Answer 1

Yes and no. It's perfectly possible to use PyBrain's optimization methods on arbitrary functions, but as those methods are 'black-box' methods, they do not make use of supplied derivatives (nor do they try to compute them).

For example, you can optimize the value of an arbitrary function f (starting at a point x0) like this:

from pybrain.optimization.distributionbased.xnes import XNES
f = lambda x: x[0]**10+sum(x[1:]**2)
x0 = [1,2,3]
print XNES(f, x0, minimize=True, 
           desiredEvaluation=1e-10, maxEvaluations=1000).learn()

Note that if you use arbitrary functions, it is recommended that you make it explicit whether it should be minimized or maximized, and how long it should be run (in terms of number of evaluations or desired fitness). I used here the exponential NES algorithm, but you can simply replace the type of algorithm class with your preference, e.g. CMA-ES, PSO, etc.

Theano + PyBrain, it is possible to combine them?

Subscription:

Related questions