|
I'd like to find a function A(I) such that B(A(I)) is minimised.
My current plan is to represent A with a neural network and find the weights using gradient descent. Anybody got a better idea? I don't really know much maths beyond high school level, so please be gentle with me. |
|
Your question is very abstract and what method to use depends a lot on the nature of the variables I and the second function B. A neural network seems a sensible choice if you are indeed able to obtain derivatives of B with respect to the output of A. You can indeed use gradient descend to optimize the parameters but there are various alternatives like quasi-newton methods and nonlinear conjugate gradient for example that might work better. If B is some kind of well known regression loss like the mean squared error, you could give any regression method (Gaussian process regression for example) a shot. For classification there are also numerous possibilities other than neural networks that can perform better under certain circumstances. If A can be assumed to be linear you might be better off using a linear regression or classification method. The bottom line is that it is difficult to suggest alternatives with so little knowledge about the problem. Given the information you supplied so far I would also use a neural network because of the available gradient information but if more details are known there might be better alternatives. @Philemon Thanks for your answer. I'm not at liberty to reveal the real problem. A is not linear. B is not a kind of regression loss: It is more like an integral over lots of A(I). Do you have specific questions that I could answer that would help?
(Jan 12 '11 at 08:46)
mjb
I wonder what causes the secrecy but some things that come to mind are: How many values of I do you have to train on? Are the observations of I independent (things become different if I is some sort of sequence data for example)?
(Jan 12 '11 at 09:57)
Philemon Brakel
I is a dummy variable. B depends on the entire range of I. A correct formulation of A probably has only one minimum.
(Jan 12 '11 at 11:53)
mjb
It is difficult to abstract a problem away from a real problem even if you have a strong math background. I have a reasonably strong math & CS background, and I have a hard time modeling & describing abstract variations on problems to others. With a limited math background this becomes even more difficult. You'd be better off if you could re-cast your problem into an example that allows you to discuss it without revealing the true nature of your original problem. As Phil said, your statement is a little too abstract. -Brian
(Jan 12 '11 at 14:32)
Brian Vandenberg
There are many different methods for regression, classification and optimization. When the problem is so general they might all be that 'best' method. You might actually be better off consulting an introductory book on Machine learning or trying different methods from a standard toolbox to just see what works well. This way you don't have to worry about disclosing too much information either.
(Jan 13 '11 at 04:00)
Philemon Brakel
|
|
If the variables in I do not follow a nice form that allows 'solving', try genetic algorithms, which do well for messy problems. |
An even simpler idea is to learn a linear combination. Add 0/1 valued variable per category per variable. Now, each variable has a weights, and you get final value by multiplying weights by variable values. Train is by gradient descent as well. Since A has 2 values, you need 2 sets of weights
I am not quite sure if you really mean the question as you posted it. The function B only depends on A(I), not on A and I separately? So this is not a loss on a function but just some function on some real space? Well then you can simply find the global minimum of B and set A to be constant to that value, independent of I....