Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Kernel] Flash attention 2 #275

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

[Kernel] Flash attention 2 #275

wants to merge 14 commits into from

Commits on Sep 26, 2024

  1. Added flash attention kernel

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    2f42f74 View commit details
    Browse the repository at this point in the history

  2. Added monkey patching for SDPA

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    8412147 View commit details
    Browse the repository at this point in the history

  3. Added and passed tests in test_mini_models

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    5731707 View commit details
    Browse the repository at this point in the history

  4. Small changes to the fa ops

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    638882f View commit details
    Browse the repository at this point in the history

  5. Added and passed tests in no_logits (skipped in multi)

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    e04dc87 View commit details
    Browse the repository at this point in the history

  6. Unit tests, nearly all passed

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    23d0df9 View commit details
    Browse the repository at this point in the history

  7. Benchmarking

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    6c03dcf View commit details
    Browse the repository at this point in the history

  8. Reduced test numbers

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b32cf9f View commit details
    Browse the repository at this point in the history

  9. Formatted the style

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b213b0e View commit details
    Browse the repository at this point in the history

  10. Further style

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    c316b87 View commit details
    Browse the repository at this point in the history

  11. Changed the test threshold in accordance with FA repo

    @remi0r
    remi0r committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    15cfc63 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Merge branch 'main' into main

    @lancerts
    lancerts authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    eb2683b View commit details
    Browse the repository at this point in the history

  2. Changed the test threshold in accordance with FA repo

    @remi0r
    remi0r committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    6a31c03 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. Merge branch 'main' into main

    @lancerts
    lancerts authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    cc71c5d View commit details
    Browse the repository at this point in the history