|
This is a pretty basic question, but better to have this right. Is there a fundamental difference between a DAG, a Belief Network and a Bayesian Network? According to the Wikipedia Article, they are pretty much the same, but I just wanted to be sure before writing something fundamentally wrong. Thanks |
|
A "belief network" and a "Bayesian network" are the same thing, they are both directed acyclic graphs that represent a probability distribution on the nodes where each node n represents a factor in the joint distribution corresponding to the probability that n takes on a value given the values of its parents in the graph. "Deep belief nets" usually refer to a specific sort of graphical model with (usually) binary stochastic variables in all layers, with directed connections between every adjacent pair of layers except the last, which form an undirected associative memory. The parameters are usually learned by recursively learning restricted Boltzmann machines at each layer and then globally fine-tuning in some fashion. Thanks, I did have some confusion since they were solved using RBM's and did not find an specific reason for that.
(Jul 23 '11 at 03:09)
Leon Palafox ♦
|
The answer is NO. You've really hit the nail on the head with this one, in identifying the remarkable diversity and possibilities of graphical models (the only real distinction to make is directed versus undirected, or chain graphs which mix the two; these require different methods but are fundamentally to solve but are not fundamentally different). This is why people mash graphical models together in such crazy ways. Its worth noting that the Deep Belief Network was originally developed with a quite unusual inference scheme called contrastive divergence; however this is not strictly necessary. Bayesian Networks and DAGs are the same though.
Just a clarification: contrastive divergence is an approximate learning procedure for RBMs, not an inference scheme. Inference in RBMs (and in DBNs, due to the particular form they take) is trivial.
Why is inference in DBNs trivial? I am pretty sure it is not. Usually a heuristic based on mean field approximations is used. But as far as I can see, this is far from being exact inference.