In a gaussian mixture model, I don't really understand how to update the covariance matrix s_j associated to each cluster. Is the following code correct (to see if I've understood how it works):

If I understand, we should repeat the following code algorithm until convergence:

X = {x1, ..., xn} set of data-points
M = {m1, ..., mk} set of k means choosen randomly from X
S = {s1, ..., sk} set of k covariance matrix, each si is choosen to be the identity matrix

for i from 1 to n
{
    for j from 1 to k
    {
        compute Pij = pdf(xi | mj, sj) / SUM_l pfd(xi | ml, sl)
    }
}

for i from 1 to n
{
    for j from 1 to k
    {
        mj = mj + (Pij * xi) / SUM_l Plj // update the mean
        // here how to update sj ? covariance matrix
    }
}

asked Nov 09 '12 at 09:17

shn's gravatar image

shn
462414759

You don't say, but it sounds like you are trying to use EM to fit the mixture model. My suggestion is to see if you can figure out the details in the one dimensional case with just a single feature.

(Nov 10 '12 at 07:33) Art Munson

One Answer:

The update is analogous to the way you update the mean but you need those new means first so it cannot happen in the same loop. It would be something like sum_i( Pij * (xi - mj) * (xi - mj).T ) / sum_l plj. Hope this helps.

answered Nov 20 '12 at 07:23

Philemon%20Brakel's gravatar image

Philemon Brakel
2445103560

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.