Start with a blank approach to an NLP task, or read a lot before starting?

You will find that your intuitions are likely to be misleading when working on natural language processing tasks. Human beings have both built-in neural hardware for language processing and huge amounts of experience with the world and with using language. Your software will have none of that.

So...read as much as you can about previous approaches to solving the problem you care about. Look for repeating themes across a wide range of work, since an individual researcher will often be incorrect about what the actually reasons for success or failure of a particular technique were.

Think about the problem, ask questions, try to find answers with the help of papers, discussions with colleagues, etc. Then start implementing and experimenting.

My approach to learning new things:

• Formulate the problem
• Try to find solution without any special prior knowledge (do not code at this level!)
• Read papers/learn/reproduce some results, maybe
• Try to solve/write code
• Read papers/learn/...
• Try to solve again/write code
• .....
  

This is iterational process and it converges )

Just as a complement to Alexandre Passos excellent list. If you do start w/o reading anything, you'll have a better appreciation for what you do read. e.g. if you've banged your head trying to tackle a problem, then read an article or chapter on how the current way to tackle the problem works, you'll appreciate it more. You'll understand some of the issues, because you met them before you were exposed to the theory and the general approaches.

Surely it depends on how much experience you have with natural language processing. If you already have a lot, it can be extremely instructional to try and fail with the tools you know, because when you finally read up and learn a new one, you know exactly why you need it, and why (in practical terms) it works better than anything you knew already.

If you don't have a lot of experience, you risk stumbling upon a correct or high-test-accuracy answer by accident or because of a bug, and not really learning anything.

Both could work, and both induce different biases.

Not reading anything makes it easier to (among other things):

• Formulate the problem in a subtly wrong way that makes no sense linguistically or leads you towards bad ways of solving it
• Not use standard evaluation metrics/not optimize them
• Ignore easy solutions to possible subproblems (for example, trying to resolve coreference without doing named entity recognition or part of speech tagging or chunking)
• Fall into tempting dead ends/think a standard baseline is a very cool new technique
• Think a special case of the problem you find interesting is interesting to the general community while it is trivial/uninteresting/loosely formulated/etc
• Think that a good design decision that most techniques make is bad, while an early paper actually showed the alternative was a lot worse.

(I've made all of these, btw)

Reading everything you can get your hands on, on the other hand, might lead you to fail to see different approaches to solving the problem that can substantially improve upon existing techniques.

I guess, overall, it's good to have an acquaintance with previous work but also keep in mind that it does not necessarily encompass the totality of solution space (after all, for example, interesting linear binary classifiers, such as confidence-weighted algorithms or budgeted stochastic svms, are still being developed).

Think first, program later. Great advice, seldom listened to.