Revision history[back]
click to hide/show revision 1
Revision n. 1

Jun 26 '11 at 20:26

zaxtax's gravatar image

zaxtax
1051122545

Averaging weights in a distributed online learning problem?

So, I am trying to distribute a massive structured prediction problem. I am treating this as an online learning problem where I adjust the weights as I encounter training points. I have a large number of data points and want to distribute the training across a cluster. I recall reading a paper that says I can partition my training data, train each batch separately and average the weights. Is this actually true? If this is true, are there any papers which include a proof of why this works?

click to hide/show revision 2
Revision n. 2

Jun 26 '11 at 21:06

zaxtax's gravatar image

zaxtax
1051122545

Averaging weights in a distributed online learning problem?

So, I am trying to distribute a massive structured prediction problem. I am treating this as an online learning problem where I adjust the weights as I encounter training points. I have a large number of data points and want to distribute the training across a cluster. I recall reading a paper that says I can partition my training data, train each batch separately and average the weights.

Is this actually true? If this is true, are there any papers which include a proof of why this works?

click to hide/show revision 3
Revision n. 3

Jun 27 '11 at 00:40

zaxtax's gravatar image

zaxtax
1051122545

Averaging weights in a distributed online learning problem?

So, I am trying to distribute a massive structured prediction problem. I am treating this as an online learning problem where I adjust the weights as I encounter training points. I have a large number of data points and want to distribute the training across a cluster. I recall reading a paper that says I can partition my training data, train each batch separately and average the weights.

Is this actually true? If this is true, are there any papers which include a proof of why this works?

The closest paper I found was Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models by Mann et al

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.