|
It seems to me that GPU processing can fundamentally change how we do machine learning on parallel systems. I am wondering if other people have tried CUDA for say large and sparse PCA tasks or other ML problems. Are there papers that address in what cases a GPU may help? Obviously we have mahout's approach of solving ML problems by throwing it into the map-reduce process but is it even a good idea? What aspects of GPU can help us do ML more efficiently compared to CPUs? Any insights that you may have will definitely help. I have just started to look into how we can use these tools for ML. |
|
There is an interesting work on using parallel processors to speed up the Indian Buffet Process. This work is their latest paper on that topic. Zoubin said that it's a pretty new area. At least for stochastic processes. And it seems like a good opportunity to accelerate the Gibbs Sampling, which is the bottleneck of the process. I do not know whether someone has used CUDA, I would be really interested to work with someone trying to accelerate the IBP using CUDA. thanks for the pointer to the paper.
(Aug 25 '11 at 04:08)
Mark Alen
|
|
GPUs are best for fine-grained parallelism. They are used A LOT in computer vision now. It's easiest to speed up matrix operations. Dense matrix multiplication can gain a factor of 200. For example kmeans can be speeded up very easily. The same is true for gaussian mixtures with EM or variational methods. As soon as you can write something as matrix operations, it's trivially to accelerate. For example linear classifiers can be brought to the gpu very easily. There are also several implementations of SVMs: cuSVM GPU-LIBSVM GPUSVM If you want to play around with GPU computations, I suggest you check out my labs CUDA library. We are trying to make the use of GPUs as easy as possible and wrap everything up in Python - though you can directly use the C++ interface. There is also a library from Toronto, theano. Generally I try to blog about using CUDA and our library for machine learning. My colleague Hannes also sometimes blogs about using CUDA for machine learning, for example doing neares neighbours. Hope that helped! Oh I noticed GPU-LIBSVM just does the kernel computation on the GPU. This is not really doing the SVM on the GPU. I often do that myself, for example to compute very big RBF kernels. But you can just precompute the kernel and give it to any standard SVM package.
(Aug 25 '11 at 05:48)
Andreas Mueller
Indeed, theano is far more useful than it seems from just looking at it. Essentially it's a compiler from a model specification in python to optimized GPU (or CPU, it's agnostic) code that can both compute the functions you specified and compute the gradient automatically. So you can, for example, use the GPU to compute the gradients of a model and yet optimize this model using the fortran implementation of L-BFGS-B, and use the result in intuitive python code. I think they also have some support for multiple GPUs.
(Aug 25 '11 at 08:16)
Alexandre Passos ♦
I think theano is very useful, but it was to complex for me. It has a LOT of abstraction. As you said, it basically has a compiler. I am happier with our library that you can use more or less like numpy. Theano is quite probably more efficient, though.
(Aug 25 '11 at 10:09)
Andreas Mueller
In addition to opensource libraries, there's also Jacket and libjacket, from AccelerEyes. The former is a Matlab thing, the latter is a (C?/)C++ thing.
(Aug 25 '11 at 16:28)
Brian Vandenberg
|
