Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more support for N dimensions #3955

Open
AlexanderSinn opened this issue May 23, 2024 · 4 comments
Open

Add more support for N dimensions #3955

AlexanderSinn opened this issue May 23, 2024 · 4 comments

Comments

@AlexanderSinn
Copy link
Member

Some codes, such as HiPACE++, use a different number of dimensions in different parts of the code. Currently, AMREX_SPACEDIM has to be chosen as the maximum number of dimensions used, with the other places in the code not utilizing the extra dimensions by assigning a size of 1 to them. This can be very confusing, and in the case of ParallelFor and Array4 can lead to a slight performance loss. Additionally, it might be useful to have more than 3 dimensions sometimes. For example, a 3D loop with tiling on GPU could be implemented with a 6D ParallelFor.

This can be achieved by replacing the AMREX_SPACEDIM macro with a template parameter that specifies the number of dimensions.

I would volunteer to start working on this by converting IntVect to IntVectND<> and adding using IntVect = IntVectND<AMREX_SPACEDIM>;. Currently IntVect relies hevily on AMREX_D_TERM. This can be replaced by sfinae, if constexpr or for loops that will usually get unrolled since the bounds are known at compile time.

Later on, the same can be done for other types such as RealVect, Box, Geometry, Array4, as well as ParallelFor.

Even if not everything will be converted (BoxArray, MultiFab might require too much effort), just having N-dimensional indexing would be very useful.

@WeiqunZhang
Copy link
Member

Maybe we should have a meeting discussing this.

@AlexanderSinn
Copy link
Member Author

Good point, maybe at the next WarpX+AMReX development meeting?

WeiqunZhang pushed a commit that referenced this issue Jun 12, 2024
## Summary

As described in #3955, this PR converts IntVect to the n dimensional
IntVectND and adds
`using IntVect = IntVectND<AMREX_SPACEDIM>;`. Additionally, deduction
guides, support for structured bindings and the helper functions
`IntVectCat`, `IntVectSplit` and `IntVectResize` were added to
IntVectND.

## Additional background

Using structured binding support of `amrex::GpuTuple`, the following
code should work (on GPU):
```C++
int i, j, k, n;
...
// amrex::IntVectND<4>
amrex::IntVectND iv1 (i, j, k, n);
// amrex::IntVectND<8>
amrex::IntVectND iv2 = amrex::IntVectCat(iv1, iv1);
// ... = amrex::GpuTuple<amrex::IntVectND<3>,amrex::IntVectND<2>,amrex::IntVectND<2>,amrex::IntVectND<1>>
auto [iv3, iv4, iv5, iv6] = amrex::IntVectSplit<3,2,2,1>(iv2); 
// int, int, int = amrex::IntVectND<3>
auto [i2, j2, k2] = iv3;
// int = amrex::IntVectND<1>
auto [n2] = iv6;

assert(i==i2 && j==j2 && k==k2 && n==n2);
```
@AlexanderSinn
Copy link
Member Author

AlexanderSinn commented Aug 13, 2024

Now that ParallelFor is N dimensional, I think a base level of functionality for N dimensions is provided. Below, I listed some additional classes that could be converted to N dimensions similarly to IntVect/Box/ParallelFor. I have not started on any of these, so anybody can feel free to contribute.

@BenWibking
Copy link
Contributor

BenWibking commented Oct 18, 2024

@AlexanderSinn @WeiqunZhang Is there any thinking on adding an ArrayND object to generalize Array4? This would be particularly useful for interop with external codes/libraries that use std::mdspan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants