Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Block writing refactor #115

Merged
merged 24 commits into from
Aug 23, 2023
Merged

Block writing refactor #115

merged 24 commits into from
Aug 23, 2023

Commits on Aug 17, 2023

  1. covariance: rename difference row start/end vars, make non-full windo… 

    …ws `nan`
    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    65a9216 View commit details
    Browse the repository at this point in the history

  2. start _blocks module, move compute_out_shape to _blocks

    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    38baec0 View commit details
    Browse the repository at this point in the history

  3. start making tests and moving from io to _blocks

    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    963664b View commit details
    Browse the repository at this point in the history

  4. make default stack.block_shape 2048. removed shape guessing

    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    0efd7fb View commit details
    Browse the repository at this point in the history

  5. replace block_size_gb with block_shape

    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    d01c828 View commit details
    Browse the repository at this point in the history

  6. add the old script for making GLRT cutoff values

    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    76eb942 View commit details
    Browse the repository at this point in the history

  7. get BlockManager laid out for different in/out block stages

    @scottstanie
    scottstanie committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    9badc94 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. remove max_bytes options from iteration/arguments

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    075d481 View commit details
    Browse the repository at this point in the history

  2. refactor blockwise IO in single.py 

    move `setup_output_folder` to `single.py`, start consolidating output file metadata
    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    8bd5b69 View commit details
    Browse the repository at this point in the history

  3. add a debug to background Writer

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    6c0e9d9 View commit details
    Browse the repository at this point in the history

  4. fixes to BlockManager for slice sizes

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    8eed7e5 View commit details
    Browse the repository at this point in the history

  5. rename function in iter_blocks docstring

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    ccc1164 View commit details
    Browse the repository at this point in the history

  6. get first passing tests for block trimming

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    c6de6b2 View commit details
    Browse the repository at this point in the history

  7. remove old io block test, fix doctest

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    75846bb View commit details
    Browse the repository at this point in the history

  8. add changelog entry

    @scottstanie
    scottstanie committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    0923541 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. remove deleted code, add full res output to test

    @scottstanie
    scottstanie committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    d40a516 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into block-writing-refactor

    @scottstanie
    scottstanie committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    3c90102 View commit details
    Browse the repository at this point in the history

  3. rename trimmed to trimming in test

    @scottstanie
    scottstanie committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    4773b43 View commit details
    Browse the repository at this point in the history

  4. fix block shape reference in script

    @scottstanie
    scottstanie committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    3852b4c View commit details
    Browse the repository at this point in the history

  5. add test for basic cli config

    @scottstanie
    scottstanie committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    eb157f9 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. tests for missing data, fix block iteration bounds

    @scottstanie
    scottstanie committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    659e11a View commit details
    Browse the repository at this point in the history

  2. fix ps multilook function, add test

    @scottstanie
    scottstanie committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    909c9a1 View commit details
    Browse the repository at this point in the history

  3. add back in ps, set n_workers 

    GHA has `os.cpu_count < 1`?
    @scottstanie
    scottstanie committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    c4aee96 View commit details
    Browse the repository at this point in the history

  4. try isce first

    @scottstanie
    scottstanie committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    0147e34 View commit details
    Browse the repository at this point in the history