topic Difference between quasi-Newton method and dual quasi-Newton method? in SAS/IML Software and Matrix Computations

Difference between quasi-Newton method and dual quasi-Newton method?

KIMDW — Fri, 05 Jan 2024 03:53:39 GMT

Hi all,

I am working on writing a paper using glimmix. As I was investigating the dual quasi-Newton method used in glimmix, I became curious about the differences between the dual quasi-Newton method and the quasi-Newton method. Additionally, could anyone provide any papers related to the dual quasi-Newton method, please?

Re: Difference between quasi-Newton method and dual quasi-Newton method?

Rick_SAS — Fri, 05 Jan 2024 14:10:03 GMT

Recommending books is always difficult since we don't know which ones you have access to. Duality is described in most introductory textbooks on optimization, usually after the text describes convex optimization. One reference is Boyd and Vandenberghe (2004) . The book is available online at https://web.stanford.edu/~boyd/cvxbook/

When performing quasi-Newton iterations, it can be useful to solve a "dual problem" that has the same solution, but the dual formulation contains additional useful properties such as convexity which makes it easier to solve. I direct you to the online course notes by Ryan Tibshirani at Carnegie Mellon University: https://www.stat.cmu.edu/~ryantibs/convexopt-F15/lectures/

All the lectures are good, but I specifically recommend:

Lecture 11: Duality in General Programs
Lecture 13: Duality Uses and Correspondences
Lecture 14: Newton's Method, especially Slides 22-23 and the references on Slide 24.

Depending on your background and interests, you might wish to browse the other lectures.

Re: Difference between quasi-Newton method and dual quasi-Newton method?

Rick_SAS — Thu, 11 Jan 2024 16:17:53 GMT

Do you need additional information? If the previous response answers your question, please mark it as "Solved" and close this thread. Otherwise, let us know what else you need.

Re: Difference between quasi-Newton method and dual quasi-Newton method?

KIMDW — Fri, 12 Jan 2024 10:56:25 GMT

Thank you for your help.

You've been very helpful.