Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Binary quantization #82

Merged
merged 61 commits into from
Sep 19, 2024
Merged

Binary quantization #82

merged 61 commits into from
Sep 19, 2024

Commits on Jul 1, 2024

  1. First draft

    @Kerollmops @irevoire
    Kerollmops authored and irevoire committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    3659904 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. prepare for the new distance trait

    @irevoire
    irevoire committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    692f1ef View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. re-implements display for all kind of nodes taking the distance into … 

    …account
    @irevoire
    irevoire committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    27ed4d4 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. everything compiles and the api is clean, need work on the tests to e… 

    …nsure it works
    @irevoire
    irevoire committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    3315465 View commit details
    Browse the repository at this point in the history

  2. wip

    @irevoire
    irevoire committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    a27a238 View commit details
    Browse the repository at this point in the history

  3. fix the formatter and add a few comments

    @irevoire
    irevoire committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    3634125 View commit details
    Browse the repository at this point in the history

  4. rewrite the iterator of binary quantized slices

    @irevoire
    irevoire committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    3576a88 View commit details
    Browse the repository at this point in the history

  5. fix the size of the iterator

    @irevoire
    irevoire committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    facc857 View commit details
    Browse the repository at this point in the history

  6. fix spaces

    @irevoire
    irevoire committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    15a13ff View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. remove the now useless craft_owned_unaligned_vector_from_f32 function… 

    … from the distance trait
    @irevoire
    irevoire committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    e4847d2 View commit details
    Browse the repository at this point in the history

  2. get rid of the drain

    @irevoire
    irevoire committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    ddb2304 View commit details
    Browse the repository at this point in the history

  3. fix the spaces tests

    @irevoire
    irevoire committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    903ec0b View commit details
    Browse the repository at this point in the history

  4. fix the spaces again

    @irevoire
    irevoire committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    ae65345 View commit details
    Browse the repository at this point in the history

  5. add a comment explaining the relevancy issue we may encounter

    @irevoire
    irevoire committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    9ab541e View commit details
    Browse the repository at this point in the history

  6. rename the vector format to vector codec

    @irevoire
    irevoire committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    7907da9 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. fix the normalized distance for the binary quantized euclidean distance

    @irevoire
    irevoire committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    47fd2bd View commit details
    Browse the repository at this point in the history

  2. add a first test of relevancy that uses autogenerated vectors

    @irevoire
    irevoire committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    4c853b8 View commit details
    Browse the repository at this point in the history

  3. add an implementation for the binary quantized manhattan distance

    @irevoire
    irevoire committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    7f004e8 View commit details
    Browse the repository at this point in the history

  4. improve lisibility of the relevancy benchmark

    @irevoire
    irevoire committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    b6ea560 View commit details
    Browse the repository at this point in the history

  5. implements a basic oversampling

    @irevoire
    irevoire committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    0e8fba2 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. compute two_means on non binary quantized distances

    @irevoire
    irevoire committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    9f66448 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Make two_means return non binary quantized distances

    @irevoire
    irevoire committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    732767a View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. add the angular distance

    @irevoire
    irevoire committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    54626c1 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. implement the oversampling

    @irevoire
    irevoire committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6290296 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. make the binary quantized distance quick again

    @irevoire
    irevoire committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    d812bd8 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. provide a specialized method to check if a vector contains only zeros

    @irevoire
    irevoire committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    f9e2b63 View commit details
    Browse the repository at this point in the history

  2. fix the euclidean distance

    @irevoire
    irevoire committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    60875eb View commit details
    Browse the repository at this point in the history

  3. fix the tests

    @irevoire
    irevoire committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    8c541ed View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. [perf] use an hashmap instead of a roaring bitmap + a vec to store th… 

    …e immutable tree/leafs
    @irevoire
    irevoire committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    d75d550 View commit details
    Browse the repository at this point in the history

  2. [perf] reduce the number of allocations by pre-allocating the size of… 

    … the hashmap in advance
    @irevoire
    irevoire committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    cefadca View commit details
    Browse the repository at this point in the history

  3. add or fix comments

    @irevoire
    irevoire committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    c96e204 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. write an simd version of the code that converts the f32 vectors to si… 

    …md vectors
    @irevoire
    irevoire committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    0945e33 View commit details
    Browse the repository at this point in the history

  2. fix the reminder while storing binary quantized vectors

    @irevoire
    irevoire committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    d4ed3e1 View commit details
    Browse the repository at this point in the history

  3. re-implement the binary quantized to f32 with SIMD

    @irevoire
    irevoire committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    343dbfb View commit details
    Browse the repository at this point in the history

  4. add tests

    @irevoire
    irevoire committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1879821 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. push leaf id in a vec instead of a roaring bitmap because it s quicker

    @irevoire
    irevoire committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    4d07b35 View commit details
    Browse the repository at this point in the history

  2. fix the f32 -> binary quantized

    @irevoire
    irevoire committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    98cb3c9 View commit details
    Browse the repository at this point in the history

  3. improve comments and types

    @irevoire
    irevoire committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    1c975e2 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. First version of SIMD on x86_64

    @Kerollmops
    Kerollmops committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f9fea3b View commit details
    Browse the repository at this point in the history

  2. Upload the AVX version of to_vec

    @Kerollmops
    Kerollmops committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    1592fe0 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2024

  1. Update the simd aarch64 to_vec_simd function with blend functions

    @Kerollmops
    Kerollmops committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    fc7a72b View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. fix the binary quantized conversions

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b72958a View commit details
    Browse the repository at this point in the history

  2. gate each simd function behind the right arch and provide a fallback … 

    …method in case no simd instructions are available
    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    0f88bfd View commit details
    Browse the repository at this point in the history

  3. fix and improve the large binary quantized test

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    149e338 View commit details
    Browse the repository at this point in the history

  4. move the unaligned vector test to their own module

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    7e8ee31 View commit details
    Browse the repository at this point in the history

  5. move the node test to the binary_quantized_tests

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    d61b7af View commit details
    Browse the repository at this point in the history

  6. fix the warnings

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    283eb2c View commit details
    Browse the repository at this point in the history

  7. move proptest to the dev dependencies

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f2f6a7e View commit details
    Browse the repository at this point in the history

  8. fix comment

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    0b21f34 View commit details
    Browse the repository at this point in the history

  9. fix typo$

    @irevoire
    irevoire committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    605b33a View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. update a test playing with -0.0 and 0.0 since -0.0 doesn't answer the… 

    … same way to is_positive under x86 and aarch64
    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    df654bc View commit details
    Browse the repository at this point in the history

  2. fix a comment

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    be0f600 View commit details
    Browse the repository at this point in the history

  3. rename + unroll loop in from_slice_neon

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    2d9eeb2 View commit details
    Browse the repository at this point in the history

  4. update reader::plot with .is_zero

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    8f6eaaa View commit details
    Browse the repository at this point in the history

  5. fix github lint

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    c4aaa44 View commit details
    Browse the repository at this point in the history

  6. clippy again

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    9c457fe View commit details
    Browse the repository at this point in the history

  7. improve the display implementation of the splitnode

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    af598ca View commit details
    Browse the repository at this point in the history

  8. making the constants in unaligned_vector::binary_quantized more expli… 

    …cits
    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    71f6659 View commit details
    Browse the repository at this point in the history

  9. fix the cfg feature gates around the simd function

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    777effb View commit details
    Browse the repository at this point in the history

  10. remove the small relevancy benchmark in favor of the new repository

    @irevoire
    irevoire committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    393a5c5 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. fix the way we change the distance

    @irevoire
    irevoire committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    ddaedd3 View commit details
    Browse the repository at this point in the history