Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use dask to run tasks #1714

Closed
wants to merge 31 commits into from
Closed

Use dask to run tasks #1714

wants to merge 31 commits into from

Commits on Sep 2, 2022

  1. Use dask to run preprocessing tasks (and disable other tasks for now)

    @bouweandela
    bouweandela committed Sep 2, 2022
    Configuration menu
    Copy the full SHA
    63b032a View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2022

  1. Improve use of dask client as suggested by @zklaus

    @bouweandela
    bouweandela committed Sep 13, 2022
    Configuration menu
    Copy the full SHA
    4792193 View commit details
    Browse the repository at this point in the history

  2. Restore sequential and parallel task run

    @bouweandela
    bouweandela committed Sep 13, 2022
    Configuration menu
    Copy the full SHA
    11b12f1 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2022

  1. Add missing compute argument to save function for multimodel statisti… 

    …cs preprocessor output files
    @bouweandela
    bouweandela committed Sep 23, 2022
    Configuration menu
    Copy the full SHA
    6faa8f6 View commit details
    Browse the repository at this point in the history

  2. Add distributed as a dependency

    @bouweandela
    bouweandela committed Sep 23, 2022
    Configuration menu
    Copy the full SHA
    49e3589 View commit details
    Browse the repository at this point in the history

  3. Restore previous API to fix tests

    @bouweandela
    bouweandela committed Sep 23, 2022
    Configuration menu
    Copy the full SHA
    54edb73 View commit details
    Browse the repository at this point in the history

  4. Add xarray as a dependency

    @bouweandela
    bouweandela committed Sep 23, 2022
    Configuration menu
    Copy the full SHA
    021b3e0 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into dask-distributed

    @bouweandela
    bouweandela authored Sep 23, 2022
    Configuration menu
    Copy the full SHA
    53aa585 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2022

  1. Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di… 

    …stributed
    @bouweandela
    bouweandela committed Oct 14, 2022
    Configuration menu
    Copy the full SHA
    d5cf4f6 View commit details
    Browse the repository at this point in the history

  2. Support configuring dask

    @bouweandela
    bouweandela committed Oct 14, 2022
    Configuration menu
    Copy the full SHA
    0f5bda8 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2022

  1. Add a suffix to output_directory if it exists instead of stopping

    @bouweandela
    bouweandela committed Oct 16, 2022
    Configuration menu
    Copy the full SHA
    52af816 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2022

  1. Fix tests

    @bouweandela
    bouweandela committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    823c731 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2022

  1. single call to compute

    @fnattino
    fnattino committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    6f5a6bf View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2022

  1. Only start cluster if necessary and fix filename-future mapping

    @bouweandela
    bouweandela committed Nov 10, 2022
    Configuration menu
    Copy the full SHA
    b455bbb View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2022

  1. Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di… 

    …stributed
    @bouweandela
    bouweandela committed Nov 21, 2022
    Configuration menu
    Copy the full SHA
    05d69f7 View commit details
    Browse the repository at this point in the history

  2. Use iris (SciTools/iris#5031) for saving

    @bouweandela
    bouweandela committed Nov 21, 2022
    Configuration menu
    Copy the full SHA
    be991f8 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2022

  1. Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di… 

    …stributed
    @bouweandela
    bouweandela committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    93b6da1 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2022

  1. Point to iris branch

    @bouweandela
    bouweandela committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    37bb757 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into dask-distributed

    @bouweandela
    bouweandela authored Dec 5, 2022
    Configuration menu
    Copy the full SHA
    2b60264 View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2023

  1. Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di… 

    …stributed
    @bouweandela
    bouweandela committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    66bdf2e View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. Work in progress

    @bouweandela
    bouweandela committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    065b8a4 View commit details
    Browse the repository at this point in the history

  2. Update branch name

    @bouweandela
    bouweandela committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    f47d6da View commit details
    Browse the repository at this point in the history

  3. Remove type hint

    @bouweandela
    bouweandela committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    6e25901 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2023

  1. Get iris from main branch

    @bouweandela
    bouweandela committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    e176745 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into dask-distributed

    @bouweandela
    bouweandela authored Apr 21, 2023
    Configuration menu
    Copy the full SHA
    d6787fa View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. Add default scheduler

    @bouweandela
    bouweandela committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    222c7e5 View commit details
    Browse the repository at this point in the history

Commits on May 11, 2023

  1. Use release candidate

    @bouweandela
    bouweandela committed May 11, 2023
    Configuration menu
    Copy the full SHA
    b760abb View commit details
    Browse the repository at this point in the history

  2. Try to install iris from PyPI

    @bouweandela
    bouweandela committed May 11, 2023
    Configuration menu
    Copy the full SHA
    6b35638 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2023

  1. Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-di… 

    …stributed
    @bouweandela
    bouweandela committed May 12, 2023
    Configuration menu
    Copy the full SHA
    cf79d20 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. Update dependencies

    @bouweandela
    bouweandela committed May 19, 2023
    Configuration menu
    Copy the full SHA
    4348953 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. Remove pip install of ESMValTool_sample_data

    @bouweandela
    bouweandela committed May 23, 2023
    Configuration menu
    Copy the full SHA
    6bf6328 View commit details
    Browse the repository at this point in the history