Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gradient Clipping #902

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Gradient Clipping #902

wants to merge 8 commits into from

Commits on Jan 26, 2024

  1. remove deprecated ftz intrinsics

    @rainiwu
    rainiwu committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    5c532ec View commit details
    Browse the repository at this point in the history

  2. suppress spurious cargo clippy warning

    @rainiwu
    rainiwu committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    fb91f13 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Merge pull request #1 from rainiwu/remove-ftz 

    Remove ftz
    @swfsql
    swfsql committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    24a8593 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. avoid conv1d bound for cudnn

    @swfsql
    swfsql committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    4e3f7c7 View commit details
    Browse the repository at this point in the history

  2. bump gemm

    @swfsql
    swfsql committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    a8bc54c View commit details
    Browse the repository at this point in the history

  3. clippy fix

    @swfsql
    swfsql committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    557687c View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. Merge pull request #2 from swfsql/avoid-ci-errors 

    Avoid ci errors
    @swfsql
    swfsql committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    1175903 View commit details
    Browse the repository at this point in the history

  2. Adds Storage and Gradient view/mutating methods; Adds grads clamping … 

    …and cliping

- Added `dfdx::nn_traits::WithGrads` trait and `dfdx_derives::WithGrads` proc macro, basead on `ZeroGrads`.
- Added  `dfdx_core::tensor::WithStorage` trait.
- Changed some methods from `Gradients`:
  - Exposed `get_mut` as `pub`.
  - Exposed `get_ref` as `pub`, and lower the requirements from `&mut self` to `&self`.
- Added gradient clamping and cliping methods.
    @swfsql
    swfsql committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    7a21ba7 View commit details
    Browse the repository at this point in the history