|
Recently there has been a lot of work in trying to induce word representations, for example: http://metaoptimize.com/projects/wordreprs/ However, all the approaches I've seen so far attempt to models just the sequential aspect of the language. For coreference, we need things like gender and plural/singular (Jane and Jack should not be in the same cluster). For sentiment, we need positive/negative/subjective words. For event tracking and temporal reasoning, we need temporal expression cues. Is there a principled way to induce this knowledge automatically? Was there any work done on this topic? |
|
I don't know of any work on inducing representations that isn't yours or that you haven't cited. However, I think there is work on inferring some of these sorts of information about some words in an unsupervised fashion. For example, the first Haghighi & Klein coreference paper induces gender and number easily enough, and I think with a large enough corpus you should be able to get this down to some features by using a similar technique (the features could be posterior probability of using certain pronouns to talk about that word, for example) For sentiment there are many ways of inducind and augmenting polarity lexicons; I like this paper, it does a nice overview and studies the results of a specific method. Again, it should be easy enough to turn this into features. I don't know of any work on temporal expression cues, however, and would like to know if any exist. Maybe it'd be interesting to see how could one use neural networks or some such variant to induce similar information as these papers I referenced, I think. For polarity, for example, one could maybe use unsupervised pretraining of RBMs with n-gram and document (or sentence) membership features on all words of a huge corpus, and then do gradient descent with a small supervised dataset to get a binary (or more finer grained) representation and hope that it does something reasonable for the other words; and for gender/number I guess one could use any word embedding method and look only at close pronouns to a word. |
