How do you choose a representative larger corpus?

A quick-and-dirty approach is:

1. Find a very large (superset) corpus that will contain many documents of the desired type.

   For example, if you are looking for subjective sentences in English, the UKWAC (a 2 billion word corpus constructed from the Web limiting the crawl to the .uk domain) would be a good superset corpus.

2. Perform retrieval over the superset corpus, using the smaller corpus. i.e. make a biased sample of it.

   For example, index the superset corpus using Lucene and, for each document in the smaller corpus, retrieve the top ten indexed documents using the small corpus document as the search query. I share source code implementing this approach in a repo called biased-text-sample.

The devil is in the details, but even a naive implementation of the above could select a better representative large corpus than uniform sampling of the superset corpus.