Unsupervised methods for word polarity detection?

The approach you described is sensible: take a seed set of words and use context distributional similarity to induce more words. Another thing I'd try is to take text which already has sentiment like movies, game, restaurant or other kinds of reviews and correlate user ratings with words. There aren't many products for which you can't find sentiment-annotated data. Lastly, for a more resource-intensive version of the bootstrapping approach you'd described, there was a good paper at this most recent NAACL: The Viability of Web-derived Polarity Lexicons that might be of interest.

@Joseph, I think the idea you propose is similar to:

Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, 21(4):315–346

The earliest work on this was by Hatzivassiloglou and McKeown (1997) who used the insight that, for English, the conjunction "and" links adjectives of similar polarity and the conjunction "but" links adjectives of opposite polarity.

Vasileios Hatzivassiloglou and Kathleen McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of the ACL, pages 174–181.

Jan Wiebe used distributional clustering to label adjectives. This is a "cluster-and-label" kind of semi-supervised learning and quite similar to @Joseph's suggestion.

Janyce M. Wiebe. 2000. Learning subjective adjectives from corpora. In Proceedings of the 2000 National Conference on Artificial Intelligence. AAAI.

In the presence of additional lexical resources like WordNet, graph based semi-supervised learning approaches can applied.

Delip Rao and Deepak Ravichandran. 2009. Semi-Supervised Polarity Lexicon Induction. In Proceedings of the EACL '09.

That's kind of cheating on your unsupervision requirement but if your data contains emoticons (smileys) these may serve as weak and noisy labels. For instance, this paper labels text by using the emoticons (J. Read, 2005. Using Emoticons to reduce Dependency in Machine Learning Techniques for Sentiment Classification).

Here is a more ngram-based approach using the same method: Twitter as a Corpus for Sentiment Analysis and Opinion Mining

It is unclear to me why you would want to be fully unsupervised in the first place. For English, there are a lot of freely available polarity lexicons that are pretty good. Also, getting seed sets (or even labeled training instances) for other languages is a relatively easy thing. Humans (even non-NLP experts) are pretty good at it, assuming that the sentiment is expressed fairly directly, as opposed to the sort of phenomena that was studied in Greene + Resnick (2009). Of course, we also have a huge amount of supervised data in terms of rated user reviews for a number of domains. Though this supervision is not direct if your goal is to learn lexicons, it is easy to imagine a number of scenarios where you could generate domain specific lexicons from it (e.g., vanilla mutual information with positive/negative reviews). Perhaps I am missing something obvious, but why would you not want to use all this easy to acquire knowledge? Even if you have a new domain with no resources, information from other domains should not be ignored, given that there are so many cross-domain sentiment terms (e.g., 'great', 'awesome', ...).

Once you have prior knowledge (via seed sets or some other polarity lexicon or some auxiliary signal) you can do something like a joint-sentiment-topic model or any number of different ways to piggy-back on this prior knowledge.

At some point, I think you will need to inject some weak prior knowledge, such as seed sets, into the model. Sentiment is often expressed via adjectives, which are distributionally similar to each other at a local level, e.g., "the shirt was great", "the shirt was bad", "the shirt was green". It might be possible to split of sentiment from non-sentiment adjectives, but positive-versus-negative adjectives will be tough since every focus word that can be 'good', can also be 'bad'. You may get something better by looking at document level co-occurrence, but even then, without a little prior knowledge to push the model in the right direction (even just with one or two words) I would imagine getting noisy results.

I have some questions about your suggestion:

1. how is the unsupervised step different from standard distributed word encoding algorithms? (like Collobert & Weston, or Mnih & Hinton)

2. I think if you're looking explicitly for polarity it should do you good to think at the sentence- and document-level, as well, since this might incorporate some information about polarity that should not be present in a k word window.

3. Isn't it weird to assume that words in the same small window have the same polarity? Shouldn't this suffer a lot from neutral words (such as stopwords, discourse markers, etc)? For example, this extract from an amazon review, "has written an ambitious, hugely human novel" is clearly of positive polarity, but no word in there except "hugely" and "human" could/should be generally interpreted as positive. I somewhat feel that a good word representation that captures polarity should be less local, ngram-like and more global, topic-like. Although in some experiments I made with sentiment classification bigrams were a lot more discriminative and easier to interpret than words. I'm not sure that this has been reproduced, however.

I can describe an approach that I use in my work and link to a good paper about the topic.

I first trained the rainbow classifier on the standard movie reviews corpus that is included in the python NLTK. Then I sorted the list of the words in the classification corpus by info gain, ie got a list of words that were the most polar between the positive and negative categories. Then I did a simple count on each word to see if it occurred more often in the positive or negative reviews.

I used these words as seed lists and followed the procedure outlined in this paper, "Large-Scale Sentiment Analysis for News and Blogs". Essentially you start with your seed lists and then recursively query the WordNet database (also included with the python NLTK) to find synonyms. The details of the process are outlined in the paper. It can be a bit involved, but it's produced decent results for me. The polarity of words can change slightly between categories (ie for technology content "speedy" is positive but for dating content it might not be) but the movie reviews provide a good general starting point with 1000 documents in each category.