|
I have heard that neural networks are very good when implementing solutions to sequential-decision tasks. However, I assume that the qualifier "sequential" exists because there must likewise be a "non-sequential" realm. Just wondering if there were some standard, canonical examples some people knew of the top of their head. Thanks :) |
|
A non-sequential decision task is binary classification: there's usually no natural order in which examples arrive, and you can try to train and test on the whole data set at once. A sequential decision task is reinforcement learning: given a position on the maze, what does your tiny robot do. This clearly involves a lot of decisions in sequence, specially if your robot does not know the maze beforehand. There are things in the middle, such as online (bandit) classification, where the examples arrive sequentially but they are iid-ish, and the ad placement problem, where training is usually done offline but earlier decisions affect which training data you get. Would you then say that a neural network (that corresponds to an agent in a game) that has sensor data feeding into its input neurons, and certain actions corresponding to each output neuron, a non-sequential model? I'm confused because that's what it would seem like, but I always hear that example used when people are referring to "sequential" decision tasks....
(Dec 22 '11 at 11:51)
gangsta
The neural network playing a game from sensor data is clearly a sequential task. For example, if it does a good thing later on it should propagate the reward back to its earlier decisions that allowed it to get to that state. That's sequential decision making.
(Dec 22 '11 at 14:19)
Alexandre Passos ♦
|
|
I feel that 'sequential decision' refers to two things. It could imply that the decisions are dependent on each other as opposed to the independent decision making. Secondly, it also implies that the decisons are made one after the other. You are not allowed to go back and change your mind. You have to make each decision promptly, instead of waiting until all evidences are collected to make the entire sequence of decisions. The second is relatively a narrower definition. |
