|
I've read many papers which suggest generating pairs if someone clicked on document and not clicked on some previos documents. It reasonable since user scanned not-clicked documents. But what should I do if my relevance function is already relevant in such case I will consider only mistakes of my relevance function and ignore all good examples. I don't think training on only error corpra as relevance universe can lead to good results. So I'm wondering how can I generate training pairs from correct sorting by baseline relevance function(currently on website)? |
I've read your question a couple of times and I just don't understand it. Sorry. Maybe link to a paper solving the kind of question you're interested in, and frame it in terms of the work presented therein?
Here are three relevant papers on the topic.
Can you learn from ranking functions from clicks? http://research.microsoft.com/en-us/people/zhichdou/ct_ranking_paper.pdf Use randomization in your experiment, randomly twiddle adjacent results: http://www.cs.cornell.edu/people/tj/publications/radlinski_joachims_06a.pdf User click modelling: http://users.cis.fiu.edu/~lzhen001/activities/KDD2011Program/docs/p1388.pdf