Summary

Email me your pitch and how you need help monetizing data.
If I like your pitch, I’ll give you a free consultation on data strategy (NLP, ML, business intelligence, etc.)
Afterwards, if we both think that I can add value to your business, we can talk about a longer-term relationship.

You should forward this blog post to any friend who could use this information.

What is data strategy?

Do you know how to monetize the data you have? How can you improve monetization using other data available to you? How do you transform your data into actionable business intelligence?

I can help you shape your data strategy, your long-term plan for how your business will capture, process, and monetize data. For example, data strategy can help you in the following circumstances:

• You don’t know who your individual users are or what they want, so you can’t effectively target ads.
• You don’t know what user behavior on your site to track.
• You don’t know what information you should start scraping from the web, information which you could use months or years down the line.

Besides working backwards from your business goals and business assets to a viable data strategy, I can also help you with more concrete challenges in NLP and machine learning:

• How do I improve my search engine so that users don’t miss out on relevant results?
• How do I add or improve recommendation, to connect users with what they want?
• How do I scale this ML algorithm to billions of examples with millions of features?
• How do I improve the accuracy of this NLP or ML tool?

My name is Joseph Turian, and I head MetaOptimize LLC. We consult on NLP, ML, and data strategy. We also run the MetaOptimize Q&A site, where ML and NLP experts share their knowledge.

• I am a data expert, holding a Ph.D. in natural language processing and machine learning. I have a decade of experience in these topics. I specialize in large data sets.
• I’m business-minded, so I focus on business goals and the most direct path of execution to achieve these goals.
• I am also a technology generalist who has been hacking since age 10 and has programmed competitively at a world-class level.

References from clients past and present available upon request.

What is the offer?

You send me information about what you’re doing and why you think I can help you.
Bonus points if you send me your deck, so I can understand your entire business picture. You are asking me to invest valuable expertise and potentially IP in your company, so appeal to me as a potential investor.
Demerits if you send me an NDA prematurely. Uptight companies who think what they are doing isn’t protected by good execution are a turn-off. But if you must be all James Bond about it, I’ll still consider you.

If I like what you’re doing and I can budget time, we schedule a meeting (in person or over Skype) and I’ll give you a free consultation on what you’re doing.

If the initial meeting goes well, and we both see how I can add value to your business, we can decide to continue working together. I can continue to help you either:

• Advising you periodically about your data strategy.
• Building you new tools to use in your product.
• Licensing to you existing tools I’ve already built.
• Training your smart tech geeks on NLP and ML technology for you to build in-house.

Compensation accepted in the form of cash or equity or a mix of both. Pro-bono if you’re an awesome non-profit.

Why am I doing this?

• More deals is always good.
• I am a social hacker, and enjoy connecting and sharing with other entrepreneurs. I want to meet some more excellent people.
• I would like to improve my understanding of widespread challenges and pain points in data strategy. That way, I can build a product that is useful for many people.
• This is an interesting social business experiment.

Who is this offer for?

• Open-source projects looking to use NLP + ML to improve their users’ experience.
• Unfunded startups with a promising team, product, and market.
• Funded startups.
• Established companies.

What are you waiting for?

Email me your pitch and how you need help monetizing data.
Or forward this blog post to a friend who could use this information.

Summary

A pattern for persisting generators is to turn them into pickle-able class objects. This is useful when you use generators for streaming training examples.

I would also try generator_tools, which might be a more convenient alternative to the pattern I describe. I haven’t used it yet.

Generators for streaming training examples

For machine learning, python generators are a simple idiom that make it easy to generate a stream of training examples. Moreover, you can nest generators:

• The inner generator can be used to read one example at a time.
• The outer generator can be used to read examples from the inner generator until you have a full minibatch, and then yield this minibatch.

Here is some example code:

[Update: The example holds without the ALL CAPS magic variable names, “HYPERPARAMETERS”. However, I include HYPERPARAMETERS because I am including the actual code I am using. Hyperparameters are global, read-only variables that specify the particular experimental condition being tested. I can’t say that I have the best solution to this particular aspect of experimental control (hyperparameters). I might write a blog post about it in the future, to solicit feedback on improved methods. However, I have refined my current approach over several years, and I can assure you that it is far less painful than a handful of more “clean” approaches.]

def get_train_example():
    HYPERPARAMETERS = common.hyperparameters.read("language-model")

    from vocabulary import wordmap
    for l in myopen(HYPERPARAMETERS["TRAIN_SENTENCES"]):
        prevwords = []
        for w in string.split(l):
            w = string.strip(w)
            id = None
            if wordmap.exists(w):
                prevwords.append(wordmap.id(w))
                if len(prevwords) >= HYPERPARAMETERS["WINDOW_SIZE"]:
                    yield prevwords[-HYPERPARAMETERS["WINDOW_SIZE"]:]
            else:
                prevwords = []

def get_train_minibatch():
    HYPERPARAMETERS = common.hyperparameters.read("language-model")
    minibatch = []
    for e in get_train_example():
        minibatch.append(e)
        if len(minibatch) >= HYPERPARAMETERS["MINIBATCH SIZE"]:
            assert len(minibatch) == HYPERPARAMETERS["MINIBATCH SIZE"]
            yield minibatch
            minibatch = []

You can’t persist training state by pickling your generators

However, generators become problematic when you want to persist your experiment’s state in order to later restart training at the same place. Unfortunately, you can’t pickle generators in Python. And it can be a bit of a PITA to workaround this, in order to save the training state.

Pattern to workaround this annoyance

Following useful discussion on pylearn-dev and stackoverflow [1] [2], I propose the following pattern for converting generators to pickle-able class objects:

1. Convert the generator to a class in which the generator code is the __iter__ method
2. Add __getstate__ and __setstate__ methods to the class, to handling pickling. Remember that you can’t pickle file objects. So __setstate__ will have to re-open files, as necessary.

Here is the updated code, after applying this pattern:

class TrainingExampleStream(object):
    def __init__(self):
        # Set the state variables, in case pickling happens before __iter__ is called.
        self.filename = None
        self.count = 0
        pass

    def __iter__(self):
        HYPERPARAMETERS = common.hyperparameters.read("language-model")
        from vocabulary import wordmap
        self.filename = HYPERPARAMETERS["TRAIN_SENTENCES"]
        self.count = 0
        for l in myopen(self.filename):
            prevwords = []
            for w in string.split(l):
                w = string.strip(w)
                id = None
                if wordmap.exists(w):
                    prevwords.append(wordmap.id(w))
                    if len(prevwords) >= HYPERPARAMETERS["WINDOW_SIZE"]:
                        self.count += 1
                        yield prevwords[-HYPERPARAMETERS["WINDOW_SIZE"]:]
                else:
                    prevwords = []

    def __getstate__(self):
        return self.filename, self.count

    def __setstate__(self, state):
        """
        @warning: We ignore the filename.  If we wanted
        to be really fastidious, we would assume that
        HYPERPARAMETERS["TRAIN_SENTENCES"] might change.  The only
        problem is that if we change filesystems, the filename
        might change just because the base file is in a different
        path. So we issue a warning if the filename is different from what is expected.
        """
        filename, count = state
        print >> sys.stderr, ("__setstate__(%s)..." % `state`)
        iter = self.__iter__()
        while count != self.count:
#            print count, self.count
            iter.next()
        if self.filename != filename:
            assert self.filename == HYPERPARAMETERS["TRAIN_SENTENCES"]
            print >> sys.stderr, ("self.filename %s != filename given to __setstate__ %s" % (self.filename, filename))
        print >> sys.stderr, ("...__setstate__(%s)" % `state`)

class TrainingMinibatchStream(object):
    def __init__(self):
        pass

    def __iter__(self):
        HYPERPARAMETERS = common.hyperparameters.read("language-model")
        minibatch = []
        self.get_train_example = TrainingExampleStream()
        for e in self.get_train_example:
            minibatch.append(e)
            if len(minibatch) >= HYPERPARAMETERS["MINIBATCH SIZE"]:
                assert len(minibatch) == HYPERPARAMETERS["MINIBATCH SIZE"]
                yield minibatch
                minibatch = []

    def __getstate__(self):
        return (self.get_train_example.__getstate__(),)

    def __setstate__(self, state):
        """
        @warning: We ignore the filename.
        """
        self.get_train_example = TrainingExampleStream()
        self.get_train_example.__setstate__(state[0])