ADAPTIVE, LIMITED-MEMORY BFGS ALGORITHMS FOR UNCONSTRAINED OPTIMIZATION

The limited-memory BFGS method (L-BFGS) has become the workhorse optimization strategy for many large-scale nonlinear optimization problems. A major difficulty with L-BFGS is that, although the memory size m can have a significant effect on performance, it is difficult to know what memory size will work best. Importantly, a larger m does not necessarily improve performance, but may, in fact, degrade it. There is no guidance in the literature on how to choose m. In this paper, we briefly review L-BFGS and then suggest two computationally efficient ways to measure the effectiveness of various memory sizes, thus allowing us to adaptively choose a different memory size at each iteration. Our overall numerical results illustrate that our approach improves the performance of the L-BFGS method. These results also indicate some further directions for research.