I have implemented some gradient descent optimizers for linear regression :
- Vanilla gradient descent
- Batch gradient Descent (with momentum, nesterov acceleration..)
- Adagrad
- RMSPROP
- Adam ...
TODO :
- Add level curve plot
- KSGD
- More generic implementation (polynomial)
- Evolution of error on the fly
- Newton method (logistic regression, but maybe in a new repo)