Difference between quasi-Newton method and dual quasi-Newton method?

[KIMDW]

Hi all,

I am working on writing a paper using glimmix. As I was investigating the dual quasi-Newton method used in glimmix, I became curious about the differences between the dual quasi-Newton method and the quasi-Newton method. Additionally, could anyone provide any papers related to the dual quasi-Newton method, please?

[Rick_SAS]

Recommending books is always difficult since we don't know which ones you have access to. Duality is described in most introductory textbooks on optimization, usually after the text describes convex optimization. One reference is Boyd and Vandenberghe (2004) .  The book is available online at https://web.stanford.edu/~boyd/cvxbook/ 

When performing quasi-Newton iterations, it can be useful to solve a "dual problem" that has the same solution, but the dual formulation contains additional useful properties such as convexity which makes it easier to solve. I direct you to the online course notes by Ryan Tibshirani at Carnegie Mellon University: https://www.stat.cmu.edu/~ryantibs/convexopt-F15/lectures/ 

All the lectures are good, but I specifically recommend:

• Lecture 11: Duality in General Programs
• Lecture 13: Duality Uses and Correspondences
• Lecture 14: Newton's Method, especially Slides 22-23 and the references on Slide 24.

Depending on your background and interests, you might wish to browse the other lectures.

[Rick_SAS · Accepted solution]

Do you need additional information? If the previous response answers your question, please mark it as "Solved" and close this thread. Otherwise, let us know what else you need. 

[KIMDW]

Thank you for your help.

You've been very helpful.