From 25ef502b4be5ea08f02c53581e7d24900a70b8e0 Mon Sep 17 00:00:00 2001 From: mloubout Date: Wed, 4 Aug 2021 15:46:56 -0400 Subject: [PATCH] README --- README.md | 42 +++++++++++++++++++++++++++++++++++ examples/modeling_basic_2D.jl | 6 ++--- test/debug.jl | 3 --- test/runtests.jl | 0 4 files changed, 45 insertions(+), 6 deletions(-) delete mode 100644 test/debug.jl create mode 100644 test/runtests.jl diff --git a/README.md b/README.md index e69de29..54e4230 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,42 @@ +# JUDI4Azure.jl + +This packages implements serverless parallelism on Azure (Azure batch) for JUDI using [AzureClusterlessHPC](https://github.com/microsoft/AzureClusterlessHPC.jl). + +Using this package overwrite the task parallelism in [JUDI](https://github.com/slimgroup/JUDI.jl) and will therefore throw the corresponding warnings. These warnings can be safely ignored, however, `JUDI` and `JUDI4Azure` should not be used together. The current implementation doesn't allow to switch between azure and conventional parallelism, it is left to the user to know which resources they will use. + +## Usage + +To use this package, simply replace `using JUDI` by `using JUDI4Azure` at the top of your script. All JUDI functionnalities are reexported to make your script compatible. + +Once `JUDI4Azure` imported, you can use its main functionnality to start an Azure batch pool to use as your serverless remote taks farm. To start a batch pool with `2` nodes and `4` threads per node run: + +```julia +nworkers = 2 +init_culsterless(nworkers; credentials=creds, vm_size="Standard_E2s_v3", pool_name="PoolTest", verbose=1, nthreads=4) +``` + +where: +- `creds` is the path to your credentials JSON file (see [credentials](https://microsoft.github.io/AzureClusterlessHPC.jl/credentials/) for informations) +- `vm_size` is the Azure VM you want to use. This VM need to be available in your bqatch account +- `pool_name` is the of the batch pool of nodes +- `verbose` controls [AzureClusterlessHPC](https://github.com/microsoft/AzureClusterlessHPC.jl)'s verbosity +- `nthreads` defines the number of OpenMP threads to use on each node + +## Examples + +A simple example is available at [examples/modeling_basic_2D.jl](https://github.gatech.edu/mlouboutin3/JUDI4Azure.jl/blob/master/examples/modeling_basic_2D.jl). This example si the verbatim copy of the corresponding JUDI example, with the exception of the Azure setup at the top, and shows the seemless usxability of this package. + +## Future work + +The following extensions are currently in developpement: + +- GPU support. [JUDI](https://github.com/slimgroup/JUDI.jl) support's GPU offloading through [Devito](https://github.com/devitocodes/devito) and VM/Image/container support is beeing added +- More flexibility allowing to switch between conventional julia parallelsim and Azure +- Some functionnalities are currently not supported yet (Extended source, TWRI), these will be added as well + + +# Author + +This software is develloped as Georgia Institute of Technology as part of the ML4Seismic consortium. For questions or issues, please open an issue on github or contact the author: + +- Mathias Louboutin: mlouboutin3@gatech.edu \ No newline at end of file diff --git a/examples/modeling_basic_2D.jl b/examples/modeling_basic_2D.jl index 0d08458..fc5bbab 100644 --- a/examples/modeling_basic_2D.jl +++ b/examples/modeling_basic_2D.jl @@ -1,10 +1,10 @@ # Example for basic 2D modeling: # The receiver positions and the source wavelets are the same for each of the four experiments. -# Author: Philipp Witte, pwitte@eos.ubc.ca -# Date: January 2017 +# Author: MAthias Loubout, mlouboutin3@gatech.edu +# Date: August 2021 # -using JUDI4Azure, SegyIO, LinearAlgebra +using JUDI4Azure creds = "/home/mloubout/research/azure/clusterless_creds.json" init_culsterless(2; credentials=creds, vm_size="Standard_E2s_v3", pool_name="PoolTest", verbose=1) diff --git a/test/debug.jl b/test/debug.jl deleted file mode 100644 index d64495a..0000000 --- a/test/debug.jl +++ /dev/null @@ -1,3 +0,0 @@ -using JUDI4Azure - -init_culsterless(2; credentials="/home/mloubout/research/azure/clusterless_creds.json", vm_size="Standard_E2s_v3", verbose=1) \ No newline at end of file diff --git a/test/runtests.jl b/test/runtests.jl new file mode 100644 index 0000000..e69de29