Hi, Annealing and Homotopy seems to me work the same. They both runs a local optimization algorithm on a relaxed versions of the original problem to feed initial point for a harder version of the problem. Supposing annealing is done deterministically, is there any technical difference between them? Thanks

asked Aug 04 '12 at 11:45

Arya%20Iranmehr's gravatar image

Arya Iranmehr
106337


One Answer:

What do you mean exactly by deterministic annealing? Fully deterministic (including state updates, and if so, how?), or just the temperature schedule?

At a high level, I would say that the main difference between homotopy and annealing, besides the fact they use gradient differently (homotopy uses a full gradient with respect to the joint (parameter,'temperature') to track the solution to a fixed-point equation, annealing, if it uses the gradient at all, would use it to "approximately minimize"/sample an energy function, probably at fixed temperature), is that, at least in simulated annealing, you can often theoretically get to the desired solution by only cooling (although slowly enough).

On the other hand, the homotopy parameter alpha (which, roughly speaking, plays the role of an inverse temperature), may need to alternatively decrease and increase to guarantee convergence to the desired solution. More precisely, the homotopy argument will often guarantee the existence of a curve from (trivial-solution,alpha=0) to (desired-solution,alpha=1), but this curve may go sometimes in the direction of decreasing alpha - see http://uai.sis.pitt.edu/papers/02/p111-corduneanu.pdf figure 2 for a picture).

I am not sure how fully deterministic annealing would get around the need to 'reheat' the system to get to the optimal solution (in the absence of extra structure), but I think a 'naive approach' (alternate between slightly decreasing temperature and using current parameters as initialization to a local optimization) can definitely be made to fail.

answered Aug 08 '12 at 18:17

Theo%20Weber's gravatar image

Theo Weber
9624

Thanks for reply! so roughly speaking, the main difference is : Since homotopy methods use joint gradient of parameter and temperature they might not monotonely reduces temperature but annealing methods always reduce temperature? nice paper! could you suggest more papers/references about my question? cheers, -- Arya

(Sep 23 '12 at 14:09) Arya Iranmehr
Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.