Training subset of weights in neural networks.

I wanted to know if there is any work done on training feed forward artificial neural networks (single hidden layer) training where a subset of weights are adjusted (using gradient descent and family or evolutionary) while keeping others constant, and iterating this way in groups.

For example a set of weights A are fixed and the other set B are changed, and in the next iteration A is changed and B is fixed, and thus the process proceeds until the system does not match a stopping criteria.

training neural-networks

asked Aug 11 '13 at 08:41

phoxis
31●7●7●11

Sounds a bit like coordinate descent.

(Aug 11 '13 at 12:16) alto

@alto: thanks, good suggestion.

(Aug 19 '13 at 09:17) phoxis

2 Answers:

oldest newest most voted

There is dropout. It randomly omits part of the weights during each training sample. Improving neural networks by preventing co-adaptation of feature detectors. I don't know if it is exactly what you're looking for, but have a look.

answered Aug 11 '13 at 13:18

edersantana
155●2●5●9

Looks like it is. I need to get into it first to comment on it.

(Aug 15 '13 at 08:59) phoxis

You could also take a look at DropConnect: http://cs.nyu.edu/~wanli/dropc/

(Aug 21 '13 at 08:08) alfa

1	Dropout by Hinton group disables the output of a neuron but you may want to see into drop connect by paper here. There are code examples of CUDA also available. answered Aug 21 '13 at 09:41 Arun Kumar 286●10●10●16 This is more appropriate to me. (Aug 23 '13 at 10:26) phoxis

Subscription:

Once you sign in you will be able to subscribe for any updates here

Tags:

neural-networks ×186
training ×32

Asked: Aug 11 '13 at 08:41

Seen: 812 times

Last updated: Aug 23 '13 at 10:26

Training subset of weights in neural networks.

Subscription:

Related questions