Quickstart¶
Overview¶
This document is a work in progress, but for now, check out example code:
Gradient-based algorithms¶
Each of the gradient based algorithms has the following interface. Given a function f_df
that computes the objective and gradient of the function you want to minimize:
>>> opt = descent.GradientDescent(theta_init, f_df, 'sgd', {'lr': learning_rate})
>>> opt.run(maxiter=1000)
>>> plt.plot(opt.theta)
Proximal algorithms¶
Example code for ADMM, for solving a linear system with a sparsity penalty:
>>> opt = descent.Consensus(theta_init)
>>> opt.add('linsys', A, b)
>>> opt.add('sparse', 0.1)
>>> opt.run()
>>> plt.plot(opt.theta)
Storage¶
After calling the run
command, the history of objective values is stored on the optimizer object:
>>> opt.run(maxiter=1000)
>>> plt.plot(opt.storage['objective'])
Utilities¶
Some other features that might be of interest:
- memoization (see:
descent.utils.wrap
) - function wrapping (see:
descent.utils.destruct
anddescent.utils.restruct
) - gradient checking (see:
descent.check_grad
)
Tutorial¶
There is a tutorial consisting of jupyter notebooks demoing the features of descent at: github.com/nirum/descent-tutorial.