Neural network without hidden layers

Any model that learns an embedding for words in the usual neural language model way can be thought of as having a hidden layer with linear units. For instance, if v is a high dimensional binary unit vector (i.e. v looks like a column of the identity matrix) we can think of the table lookup in a neural language model as actually being performed by Wv where W is the table of word embeddings.

I suspect your question is really about getting neural language models to work. Assignment 1 from an undergrad course taught by Geoff Hinton at U of Toronto provides matlab code implementing a simple neural language model. (You can find the course website from Professor Hinton's web page.)

There is no way to meaningfully answer your general question about how many iterations one needs to wait before one stops training a neural net. The number of iterations depends on the learning algorithm, the initialization, the data, the neural net architecture, and the criterion used to decide convergence.

Logistic regression and linear regression can be viewed as neural nets with zero hidden layers, but as I mentioned above, for the purposes of your question I think it is best to view a neural language model that learns word embeddings as having a hidden layer formed by the embeddings themselves.