|
I am wondering what people think is the best automatic differentiation package out there these days, especially for MATLAB. However, I wouldn't mind hearing about a good package for Python or other languages. Free is a plus, but as long as it is reasonably priced I don't mind too much. Which packages can handle things in the background without you having to explicitly insert nasty expression (for instance if you differentiate in Mathematica)? I am looking for something that can handle vectorized expressions and expressions that contain operations like: trace, det, chol, inv, , ... As an arbitrary example, could it differentiate a GP likelihood dnlml/dtheta: K = f(theta); L = chol(K); alpha = L(L'y); nlml = 0.5y'alpha + sum(log(diag(L))) + 0.5nlog(2*pi); without much extra effort on the part of the programmer? How well do they handle matrix derivatives as in, dX/dY where X and Y are both matrices? This results in a 4D object. Any tips and pointers here? What are your experiences with automatic differentiation? |
|
The Python OpenOpt numerical optimization team has exposed their automatic and approximate differentiators (FuncDesigner and DerApproximator) for independent use. FuncDesigner handles the expected math functions, as well as loops and conditionals, and uses DerApproximator as a fallback for any unrecognized functions. You'll need to check whether it recognizes all your specific requirements, but this is the most convenient free solution I've found. |
|
Another option, if you were delving into native code, is CppAD. It's a C++ library for automatic differentiation. It's maintained by Brad Bell, who's pretty responsive to bugs/issues in the package. (I've only used it tangentially, though, so I can't comment personally on the ease of use.) |
Have you looked at Theano? http://deeplearning.net/software/theano/
Can I ask what you are planning to use it for?
to give an idea of things you get from automatic differentiation packages, here's first column of a derivative of an inverse of a simple 3x3 matrix valued function http://yaroslavvb.com/upload/save/inverse.png