Help with Andrew NG Machine Learning Class

Ok, a very quick primer:

In logistic regression we are trying to fit a vector of weights (W) such that the proposed function W^TX is close to the objective function Y(X).

So how do you measure this closeness? You need some sort of metric, that measures the difference between the your proposed function and your objective function.

1. This closeness measuring function is the cost function, and as a good metric, it has large values if the proposed and the objective are very different, and has small values of they are similar (or close).

So you want to find a set of weights (W) that reduce the distance between Y and W^TX as much as possible.

1. But you can only reduce this distance so much, is mathematically "impossible" to find a set of weights that generalizes for all possible W^TX, so you have to rely on a set of weights that doesn't change at every iteration of the algorithm, and this is converging.

Simple linear regression has a closed form solution, which means you can find the optimal solution regardless of any convergence. But more complex regressions do not have a closed form so you need to iterate using Newtonian methods.