Need some help clarifying my procedure using kmeans on multiple batches of data

I think you want an algorithm that looks roughly like this:

var globalMeans = new List[Vector]()
val threshold = ...
// For each batch of images you want to cluster, iterate.
for (dataset <- gather_batch_of_pictures()) {
    // Run K-Means on that batch with however many clusters you want.
    val means = kmeans(dataset, k=200)
    // For each local mean found, try to associate it with a global mean.
    for (localMean <- means) {
        // Find the closest global mean.
        var bestGlobalMean = null
        var bestGlobalSim = 0
        for (globalMean <- globalMeans) {
            val sim = similarity(globalMean, localMean)
            if (sim > bestGlobalSim) {
                 bestGlobalSim = sim
                 bestGlobalMean = globalMean
            }
        }
        // If the nearest global mean is too dissimilar, add the local mean as a new global mean
        if (bestGlobalSim < threshold)
            globalMeans += localMean
        // Otherwise add the local mean to the nearest global mean.
        else
            bestGlobalMean += localMean
    }
}

This let's you use any batch algorithm you want for the small batches you're gathering and then try to associate the local means found for the batch with global means. The global means will be created on the fly as they are needed. So at first there will be no global means and likely (if k-means did it's job right) all the local means for the first batch become global means. For the second batch, there'll likely be some clusters with local means that existed in the previous dataset. So you'll compare each local mean to existing global means, find the most similar one, and combine the local mean into that best global mean as long as their similarity is above some threshold. Setting the threshold is the hard part. You'll likely have to play around a bit to find one that works well. But with a good threshold, this will much better than just concatenating the list of local means together and over time you should see fewer and fewer global means be added as long as your dataset new distributions of features aren't dynamically being added to the dataset.

Some solutions come to mind:

1. You could just initialize the second run (or third, or etc) of the K-Means with the most recent K-Means batch you have in memory, that way you force the new SIFT features to fit in the existen clusters.
2. If you are expecting new instance or clusters being formed as you get data, you could use Dirichlet Processes instead of K-Means, with Dirichlet Processes, as you get more data, you get to create new clusters if needed.
3. Instead of using a batch approach of K-Means, use an online approach, that is, train the clusters little by little as you get new data. There must be some papers on online K-means.
4. You could look into Self Organizing Maps, which also create clusters on the go, but are kinda shady and unreliable.