Expectation of the dot product of a multivariate normal vector

Say I have a vector x ~ N(mu, Sigma). Is there an analytic formula for E[x·a], for fixed a? My first intuition is that this should be equal to mu·a, but I can only hand-wave this.

Also, the expectation of the squared norm of x, if sigma is diagonal, is something like sum_i (mu_i^2)+sigma_ii. Is there an easy way to generalize this to a non-diagonal covariance? Is there a way of arriving at this result from the answer to the previous question? Thanks a lot

calculus expectations

asked Jan 03 '11 at 11:06

Alexandre Passos ♦
18967●44●214●334

5 Answers:

oldest newest most voted

E[x.a] = sum_i x_i*a_i = mu.a. Since E[x_i] = mu_i (gaussian marginals property).

By expectation of the squared norm of x, do you mean E[x'x]? If so, then it should still be sum_i (mu_i^2)+sigma_ii even for the non-diagonal covariance. Means are additives even if the individual variables are correlated (variances aren't however): http://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables

Also see matrix cookbook (sec 8.2.2, eqn 355) to confirm the above result: http://compbio.fmph.uniba.sk/vyuka/ml/old/2008/handouts/matrix-cookbook.pdf

answered Jan 03 '11 at 12:08

ebony
181●8●10●14

Thanks for the matrix cookbook! It pretty much answers most of my other questions.

(Jan 03 '11 at 12:12) Alexandre Passos ♦

I'm not sure if I'm answering the right questions, but I think the answer to both questions is yes. My library where I'm at is a bit dated, but check Chapters Three and Four of Applied Multivariate Statistical Analysis, by Johnson and Wichern. Equation 3-31/32 is what you are looking for relative to Q1 and Equation 3-33 is a least squares answer to Q2. Chap 4 deals specifically with The Multivariate Normal if you want to get into the Max Likelihood solution.
The equations are generalized for multidimensional parameters, e.g. a'= (a_1, ..., a_n) where a_i are constants. E[a'X] = a'E[X] and V[a'X] = a' S a , where S = sample variance matrix of X and is bit ugly to present here.

FWIW this is one of the few books that focuses on vector notation in the statistical development. It is a pretty readable book and doesn't bury the reader in the theoretical gumbo.

answered Jan 03 '11 at 12:03

Aengus Robinson
215●5●11●14

Thanks. I'm actually dealing with the distribution itself, not just a sample, so there's no need for the ML solution, but it's good to know my intuitions were on the right track.

(Jan 03 '11 at 12:07) Alexandre Passos ♦

1	Answer is yes, because a.x is distributed as Normal(a'.mu,a'Sigma a), see part 3 of Theorem 14.2 in Wasserman's "All of Statistics", here are the relevant pages answered Jan 03 '11 at 12:15 Yaroslav Bulatov 1963●19●34●58