Python machine learning packages

Hi, I am one of the developers of scikit-learn . The project has recently gained steam and is moving fast thanks to new contributors. The design goals are:

• wide coverage of cutting edge algorithms with a simple to use unified API

• a permissive license for embedding (simplified BSD) and low dependencies requirements (numpy + scipy)

• optimized yet maintainable implementation using cython when useful

• scalable algorithms with dense and sparse representations of the features (useful for text classification of with ten of thousands of samples with ~100 000 features for instance).

• tooling to perform cross validation & performance evaluation across all algorithms that respect the API (duck typing)

• well tested >= 500 tests that run under 15s with a coverage of ~85% and improving (see our buildbot)

• well documented too with worked examples (could still be very much improved though)

• readable source code respecting PEP8 conventions

We don't plan to have a complete dataflow programming model as MDP does (but the two projects are collaborating (a bit) to make it easy to use scikit-learn algo in MDP nodes). It should be also possible to wrap scikit-learn models into Orange components if your users want the rich user interface of that framework (but AFAIK nobody tried so far).

Current limitations:

• the current API requires to load the training data in memory but this will evolve to handle streaming / large scale datasets (by integrating the work done by Peter in bolt)

• no command line interface: right now the user has to know basic python & numpy (the ipython shell is the most popular UI among scikit-learn devs). A generic CLI might appear in the coming months though. We will probably never offer more than CLI in terms of interface.

• very focused towards supervised learning and linear models right now. More unsupervised approaches are planned or under development though.

We also plan to provide standard feature extractors for text classification / clustering (work under way, mostly done), image classification (some basic examples), face recognition (planned) and maybe audio / speech for segmentation / classification / fingerprinting (prospective, nothing done yet). The goal is for the user to have worked example with sane default parameters to build upon and not just machine learning building blocks that require to know the inner workings to apply to a concrete use case.

Theano is a CPU and GPU compiler for mathematical expressions in Python. It combines the convenience of NumPy with the speed of optimized native machine language. For gradient-based machine learning algorithms (like training an MLP or convolutional net), Theano is from 1.6x to 7.5x faster than competitive alternatives (including those in C/C++, NumPy, SciPy, and Matlab) when compiled for the CPU and between 6.5x and 44x faster when compiled for the GPU. You can read more about it here.

scikits-learn is starting to really shape up, as others have mentioned. Other fairly mature packages include MDP and PyMVPA. If you want kernel methods, Shogun has feature-complete Python bindings and is also quite mature. There's also Elefant which I don't know much about but has been around a long time.

Let me add my own package to the list: milk and another: MDP.

I tried mlpy in it was coming out and it did not support much, which is why I ended up developing my own code. In general, scikit.learn has more cutting edge things than milk, while I focus more on performance (both speed and memory-wise) and flexibility.

Unfortunately, part of the answer depends on what you mean by "machine learning"? Classification? Clustering? Graphical models? Deep learning? No package does it all well at the moment. How large is your data?

Amund Tveit has a brief summary of Python tools for classification, with recommendations of Orange or PyML.

I only dabble in Orange several years ago(before Version 1) for text classification. It can handle thousands of samples with thousands features.
I think it's stable and mature even before Version 1. for the community, Orange has a web based forum for Orange related discussion. The authors respond quickly with nice help and suggestions.

edit: today I find this:"Python_and_Machine_Learning":slides, and video.

Don't forget MDP, probably the best for unsupervised learning with a bias towards image processing. They also have good test coverage and docs (although the site is/was a bit dated, they did a sprint and improved the looks and docs a lot)