L-BFGS for style transfer

[nsthorat]

From @carlthome on February 15, 2018 23:11

Fast style transfer with pretrained models is really cool, but the original optimization-based method presented by Gatys et al. seems like such a great fit for what deeplearn.js could provide with its WebGL integration and builtin gradient computation.

I'd really like to make interactive style transfer web apps but there's no L-BFGS optimizer in deeplearn.js. Is that a planned addition or will you focus on SGD variants?

Copied from original issue: tensorflow/tfjs-core#734

[nsthorat]

Currently we're only focusing on SGD variants of optimizers, but if you are interested in making an L-BFGS, 2nd order gradient based optimizer you can either

1. extend the Optimizer base class and override computeGradients, and applyGradients
   or
2. simply write your own optimizer without using the Optimizer class. You can compute gradients with the eager style method of differentiation (use methods from this section and then use dl.variable() and variable.assign() to do an update.

If you decide you're up for it, let us know if you run into any issues!

[rthadur]

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!