I'm trying to analyze (asymptotic if need-be) tail bounds of the posterior distribution of a bi-Gaussian latent factor model. The model is as follows.

Let (z,x) be features or covariates, and let y be the observed output. Each z is a Fx1 vector, and each x is a Dx1 vector.

Then
y ~ z' A' B x + noise
where A is a KxF matrix and B is a KxD matrix.

Each entry of A and B have zero-mean Gaussian prior distributions:
A,B ~ N(0, sigma^2)

For a specific (z,x) pair, I wish to analyze the tail of the posterior distribution of P(y|z,x,TrainingData), where TrainingData is a set of observed examples {(zi,xi,yi)}

--Analogous analysis for standard linear Gaussian models
As a point of comparison, consider the analogous linear Gaussian setting. We have:

y ~ w' x + noise

where w has a zero-mean Gaussian prior:

w ~ N(0,sigma^2)

Here, given TrainingData = {(xi,yi)}, we know that w has a Gaussian posterior distribution with some mean mu and covariance Sigma (mu being the least squares solution). Thus, for any given x, the posterior distribution of P(Y|x,TrainingData) is Gaussian with mean mu' x and standard deviation sqrt(x' Sigma x). This yields a closed-form way to reason about the tail of P(Y|x,TrainingData). For example, I know the standard deviation follows roughly sqrt(n) shrinkage as the number of training points n increases.
--End Analogous analysis

I wish to reason about the tail of P(y|x,z,TrainingData) in the bi-Gaussian latent factor model. I am most interested in analyzing things like the standard deviation or variance.

Any pointers?

asked May 22 '12 at 20:11

Yisong%20Yue's gravatar image

Yisong Yue
58631020

Be the first one to answer this question!
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.