Extracting keywords from documents for clustering

First you can do clustering without keyword extraction but extracting words or bi-grams count and then applying a TF-IDF normalization. This approach is used in this example script.

Now if you really want to extract keywords you could use a Part of Speech tagger (or even a Chunker a.k.a. Shallow Parser) for your language and extract noun phrases. You can then rank the noun phrases by averaging the IDF values of the individual words belonging to it (using a geometric mean for instance).

A simple straight method would be to:

• Count the times different words appear in the document (without stopwords), and take the probabilities of each word (#times the word appear/# of words)

• Count the times words concurred in a sentence an you can create a coocurrence matrix.

• You could argue that words that concur with words that have high counts are also keywords. You can also obtain the Pointwise mutual information between words, to see how much each word are related.

I guess this would be quite naive, but 11 documents are too few to actually do a lot on them.

There's a well-known NLP task called 'Keyphrase Extraction'. A lot of scientific papers require the authors to provide keywords or keyphrases, and this task is the automated equivalent. Most of the methods to use this allow you to lower the threshold to get more keyphrases, which might help given the small number of documents. Check out some of the papers cited by the SemEval task as well as some of the submitted systems (which are more domain dependent on scientific papers.

I'd agree with ogrisel that a TF-IDF approach might work best given your small number of documents, but if you really do need to extract keywords/keyphrases then there's a good amount of literature and the above link is a good place to start.

You can try topic modelling using LDA. Here is a tutorial for a package in R;

topicmodels: An R Package for Fitting Topic Models