Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CuGPU not defined error #244

Open
kodingkoning opened this issue Mar 11, 2024 · 15 comments
Open

CuGPU not defined error #244

kodingkoning opened this issue Mar 11, 2024 · 15 comments

Comments

@kodingkoning
Copy link

I am trying to change my simulation to use GPUs rather than just CPUs. I changed sim!(output, spec_rule,nbr_rule; proc = ThreadedCPU(), boundary=Wrap()) to use "CuGPU" instead of "ThreadedCPU", but then I get the error: nested task error: UndefVarError: CuGPU not defined

What other changed are required for CuGPU() to be recognized?

@rafaqz
Copy link
Member

rafaqz commented Mar 12, 2024

Did you try using CUDA first ?

@kodingkoning
Copy link
Author

Yes, this is with using CUDA

@rafaqz
Copy link
Member

rafaqz commented Mar 15, 2024

Ok may need DynamicGrids.CuGPU

@kodingkoning
Copy link
Author

I just tried DynamicGrids.CuCPU() as well.

I simplified it to the simplest code that gives me the error, which is:

using DynamicGrids, CUDA

DynamicGrids.CuGPU()

and results in:

ERROR: LoadError: UndefVarError: CuGPU not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:35
 [2] top-level scope
   @ ~/julia_code/test_cuda.jl:3
in expression starting at ~/julia_code/test_cuda.jl:3

The same thing happens with either DynamicGrids.CuGPU() or CuGPU() alone

@ederag
Copy link
Contributor

ederag commented Mar 18, 2024

If you are on julia-1.10.2, then I could reproduce, and it seems to require the same workaround as in #243:

(@v1.10) pkg> activate --temp
  Activating new project at `/tmp/jl_7aYPFi`

(jl_G6yIEw) pkg> add DynamicGrids#dev, CUDA

julia> DynamicGrids.CuGPU()
CuGPU{32}()

@ederag
Copy link
Contributor

ederag commented Mar 18, 2024

But if you are trying the README, you'll then face

julia> using CUDAKernels, CUDA
 │ Package CUDAKernels not found, but a package named CUDAKernels is available from a registry. 
 │ Install package?
 │   (jl_G6yIEw) pkg> add CUDAKernels 
 └ (y/n/o) [y]: 
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package CUDAKernels [72cfdca4]:
 CUDAKernels [72cfdca4] log:
 ├─possible versions are: 0.1.0-0.4.7 or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions: 0.1.0-0.4.7
 └─restricted by compatibility requirements with KernelAbstractions [63c18a36] to versions: uninstalled — no versions left
   └─KernelAbstractions [63c18a36] log:
     ├─possible versions are: 0.1.0-0.9.18 or uninstalled
     └─restricted to versions 0.9 by DynamicGrids [a5dba43e], leaving only versions: 0.9.0-0.9.18
       └─DynamicGrids [a5dba43e] log:
         ├─possible versions are: 0.21.4 or uninstalled
         └─DynamicGrids [a5dba43e] is fixed to version 0.21.4

@rafaqz
Copy link
Member

rafaqz commented Mar 19, 2024

We should really release a new version. The problem is its a breaking version and I'm not sure all the breaks make sense, and there are likely a few more on the way.

But I have been working on branches for over a year now, dev is up-to-date and actively worked on.

So using DynamicGrids#dev is highly reccomended.

@ederag
Copy link
Contributor

ederag commented Mar 19, 2024

Indeed, the breaking changes made me give up for now, as I just wanted to show your awesome package to a colleague.
Thanks to julia Manifest system, it was as simple as reopening the old notebook with julia 1.9.2.

dev is up-to-date and actively worked on.

Nice, and just to be clear, #244 (comment) was with DynamicGrids#dev and julia 1.10.2.

@rafaqz
Copy link
Member

rafaqz commented Mar 19, 2024

Yeah... my aim is to get work back on main and registered soon. But my other more general packages have been taking most of my time, plus PhD constraints.

I actually use this constantly in all of my work, but not in a stable-releases kind of way...

@kodingkoning
Copy link
Author

Thank you! I had been using an older version of Julia (1.7.3) because it was available on the cluster I was using, but I installed 1.10.2 myself, and used DynamicGrids#dev, and that's cleared up the error.

@kodingkoning
Copy link
Author

Reopening because I thought my further errors were unrelated, but now I'm thinking they are related. If it's not related, I'm glad to separate this out. Any insight on this? Am I missing some change I need to make to use the GPU? All I have changed is the value of proc to CuGPU(), and of course adding using CUDA, DynamicGrids#dev.

I'm using this code, which works with DynamicGrids and DynamicGrids#dev on the CPU, but not with the GPU.

Full error:

ERROR: LoadError: InvalidIRError: compiling MethodInstance for DynamicGrids.gpu_ka_rule_kernel!(::KernelAbstractions.CompilerMetadata{KernelAbstractions.NDIteration.StaticSize{(100, 100)}, KernelAbstractions.NDIteration.DynamicCheck, Nothing, Nothing, KernelAbstractions.NDIteration.NDRange{2, KernelAbstractions.NDIteration.StaticSize{(4, 4)}, KernelAbstractions.NDIteration.StaticSize{(32, 32)}, Nothing, Nothing}}, ::DynamicGrids.RuleData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuDeviceMatrix{Float64, 1}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuDeviceMatrix{Float64, 1}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}, Nothing, Int64, Nothing, Nothing, Nothing, Nothing}, ::Val{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}}, ::Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, ::Val{:_default_}, ::Val{:_default_}) resulted in invalid LLVM IR
Reason: unsupported call to an unknown function (call to julia.new_gc_frame)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call to an unknown function (call to julia.push_gc_frame)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call through a literal pointer (call to ijl_alloc_array_1d)
Stacktrace:
 [1] Array
   @ ./boot.jl:477
 [2] vect
   @ ./array.jl:163
 [3] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:41
 [4] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [5] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [6] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [7] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [8] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call to an unknown function (call to julia.get_gc_frame_slot)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to *)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:46
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to +)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:46
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to /)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:46
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to *)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:48
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported dynamic function invocation (call to <=)
Stacktrace:
 [1] #1
   @ ~/stochastic_competition_git/julia_code/sim_git.jl:48
 [2] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [3] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [4] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [5] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [6] gpu_ka_rule_kernel!
   @ ./none:0
Reason: unsupported call to an unknown function (call to julia.pop_gc_frame)
Stacktrace:
 [1] applyrule
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/rules.jl:259
 [2] cell_kernel!
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:223
 [3] macro expansion
   @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:67
 [4] gpu_ka_rule_kernel!
   @ ~/.julia/packages/KernelAbstractions/zPAn3/src/macros.jl:95
 [5] gpu_ka_rule_kernel!
   @ ./none:0
Hint: catch this exception as `err` and call `code_typed(err; interactive = true)` to introspect the erronous code with Cthulhu.jl
Stacktrace:
  [1] check_ir(job::GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, args::LLVM.Module)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/validation.jl:147
  [2] macro expansion
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:440 [inlined]
  [3] macro expansion
    @ ~/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:253 [inlined]
  [4] macro expansion
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:439 [inlined]
  [5] emit_llvm(job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, only_entry::Bool, validate::Bool)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/utils.jl:92
  [6] emit_llvm
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/utils.jl:86 [inlined]
  [7] codegen(output::Symbol, job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:129
  [8] codegen
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:110 [inlined]
  [9] compile(target::Symbol, job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, strip::Bool, validate::Bool, only_entry::Bool)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:106
 [10] compile
    @ ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:98 [inlined]
 [11] #1075
    @ ~/.julia/packages/CUDA/rXson/src/compiler/compilation.jl:247 [inlined]
 [12] JuliaContext(f::CUDA.var"#1075#1077"{GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}})
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/driver.jl:47
 [13] compile(job::GPUCompiler.CompilerJob)
    @ CUDA ~/.julia/packages/CUDA/rXson/src/compiler/compilation.jl:246
 [14] actual_compilation(cache::Dict{Any, CuFunction}, src::Core.MethodInstance, world::UInt64, cfg::GPUCompiler.CompilerConfig{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, compiler::typeof(CUDA.compile), linker::typeof(CUDA.link))
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:125
 [15] cached_compilation(cache::Dict{Any, CuFunction}, src::Core.MethodInstance, cfg::GPUCompiler.CompilerConfig{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, compiler::Function, linker::Function)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/U36Ed/src/execution.jl:103
 [16] macro expansion
    @ ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:359 [inlined]
 [17] macro expansion
    @ ./lock.jl:267 [inlined]
 [18] cufunction(f::typeof(DynamicGrids.gpu_ka_rule_kernel!), tt::Type{Tuple{KernelAbstractions.CompilerMetadata{KernelAbstractions.NDIteration.StaticSize{(100, 100)}, KernelAbstractions.NDIteration.DynamicCheck, Nothing, Nothing, KernelAbstractions.NDIteration.NDRange{2, KernelAbstractions.NDIteration.StaticSize{(4, 4)}, KernelAbstractions.NDIteration.StaticSize{(32, 32)}, Nothing, Nothing}}, DynamicGrids.RuleData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuDeviceMatrix{Float64, 1}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuDeviceMatrix{Float64, 1}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}, Nothing, Int64, Nothing, Nothing, Nothing, Nothing}, Val{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}}, Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Val{:_default_}, Val{:_default_}}}; kwargs::@Kwargs{always_inline::Bool, maxthreads::Int64})
    @ CUDA ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:354
 [19] cufunction
    @ ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:351 [inlined]
 [20] macro expansion
    @ ~/.julia/packages/CUDA/rXson/src/compiler/execution.jl:104 [inlined]
 [21] #_#6
    @ ~/.julia/packages/CUDA/rXson/src/CUDAKernels.jl:118 [inlined]
 [22] Kernel
    @ ~/.julia/packages/CUDA/rXson/src/CUDAKernels.jl:104 [inlined]
 [23] maprule!(data::DynamicGrids.RuleData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}, Nothing, Int64, Nothing, Nothing, Nothing, Nothing}, proc::CuGPU{32}, opt::NoOpt, ruletype::Val{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}}, rule::Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, rkeys::Val{:_default_}, wkeys::Val{:_default_})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/gpu.jl:48
 [24] maprule!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:65 [inlined]
 [25] maprule!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:17 [inlined]
 [26] maprule!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/maprules.jl:10 [inlined]
 [27] sequencerules!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/sequencerules.jl:16 [inlined]
 [28] sequencerules!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/sequencerules.jl:5 [inlined]
 [29] |>
    @ ./operators.jl:917 [inlined]
 [30] _step!(sd::DynamicGrids.SimData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, StaticRuleset{Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, Nothing, Int64, Nothing}, rules::Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:208
 [31] simloop!(output::ResultOutput{Matrix{Float64}, Vector{Matrix{Float64}}, Extent{Matrix{Float64}, Nothing, Nothing, Float64, Nothing}}, simdata::DynamicGrids.SimData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, Matrix{Float64}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::Matrix{Float64}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, StaticRuleset{Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, Nothing, Int64, Nothing}, ruleset::Ruleset{DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, fspan::UnitRange{Int64}; printframe::Bool)
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:187
 [32] simloop!
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:171 [inlined]
 [33] #runsim!#138
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:160 [inlined]
 [34] runsim!(output::ResultOutput{Matrix{Float64}, Vector{Matrix{Float64}}, Extent{Matrix{Float64}, Nothing, Nothing, Float64, Nothing}}, simdata::DynamicGrids.SimData{Tuple{100, 100}, 2, @NamedTuple{_default_::DynamicGrids.GridData{DynamicGrids.ReadMode, Tuple{100, 100}, 1, Float64, 2, Matrix{Float64}, Window{1, 2, 9, Nothing}, Wrap, Stencils.Halo{:out}, CuGPU{32}, NoOpt, Nothing, Nothing, Float64, Nothing, Nothing}}, StaticExtent{@NamedTuple{_default_::Matrix{Float64}}, Nothing, Nothing, @NamedTuple{_default_::Float64}, Nothing, UnitRange{Int64}}, StaticRuleset{Tuple{Cell{:_default_, :_default_, var"#1#2"{Float64, Float64}}, Neighbors{:_default_, :_default_, var"#3#4", Moore{1, 2, 8, Nothing}}}, DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, Nothing, Int64, Nothing}, ruleset::Ruleset{DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}, fspan::UnitRange{Int64})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:156
 [35] sim!(output::ResultOutput{Matrix{Float64}, Vector{Matrix{Float64}}, Extent{Matrix{Float64}, Nothing, Nothing, Float64, Nothing}}, ruleset::Ruleset{DynamicGrids.SimSettings{Wrap, CuGPU{32}, NoOpt, Int64, Nothing}}; init::Matrix{Float64}, mask::Nothing, tspan::UnitRange{Int64}, aux::Nothing, fps::Nothing, boundary::Wrap, proc::CuGPU{32}, opt::NoOpt, cellsize::Int64, timestep::Nothing, simdata::Nothing, kw::@Kwargs{})
    @ DynamicGrids ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:97
 [36] #sim!#136
    @ ~/.julia/packages/DynamicGrids/Z6XvQ/src/framework.jl:100 [inlined]
 [37] simulation(nrep::Int64, step::Int64)
    @ Main ~/stochastic_competition_git/julia_code/sim_git.jl:92
 [38] func(nsteps::Int64, nrep::Int64)
    @ Main ~/stochastic_competition_git/julia_code/sim_git.jl:106
 [39] macro expansion
    @ ~/stochastic_competition_git/julia_code/sim_git.jl:111 [inlined]
 [40] macro expansion
    @ ./timing.jl:279 [inlined]
 [41] top-level scope
    @ ~/stochastic_competition_git/julia_code/sim_git.jl:269
in expression starting at /u/ekoning2/stochastic_competition_git/julia_code/sim_git.jl:110

@kodingkoning kodingkoning reopened this Mar 19, 2024
@rafaqz
Copy link
Member

rafaqz commented Mar 20, 2024

Ok. So first make sure your model runs fine with e.g. proc=CPUGPU().

Then you need to be 100% sure all of the code inside your applyrule functions or rule closures are type stable and non-mutating/non-allocating. This is a GPU problem generally, rather than a DynamicGrids.jl problem - unstable or allocating code simply cant work on a GPU.

Those errors all suggest that things like this are happening. You really dont want to allocate inside applyrule even on CPU, its a huge performance hit. You will notice DynamicGrids.jl itself allocates essentially nothing over a whole simulation for this reason (e.g. using Life()). Likely fixing the allocations and type stability will mean you don't need a GPU at all, and your simulation is plenty fast on your desktop CPU!

To fix this I suggest running your CPU simulations for a few seconds using ProfileView.jl @profview.

Any red (instability) or yellow (allocations) patches in the inner function calls will mean that your model wont run on a GPU. Cthulhu.jl can help, you can click on the ProfileView.jl profile and call descend_clicked() to explore the types in your functions.

There is a method isinferred in the dev branch that will tell you if types are inferred for your rule.

As for the cluster, get your manager to add 1.10, its much better :)

(honestly here I never end up using a cluster because with no allocations my desktop is happy to run my sims a million times!)

@rafaqz
Copy link
Member

rafaqz commented Mar 20, 2024

If you paste your rules here I can probably tell you what wrong

Oops you had linked it.

This is certainly a problem, allocating an Array here for ever cell at every timestep:

upordown=sample([1,-1])

use StaticArrays.jl, and:

sample(@SVector[1, -1])

As is:

collect(1:8)

Just use 1:8 !

Basically, you should never allocate or write to any array inside a rule other than writing to a grid with e.g. add! inside a SetRule. Mostly this isn't a problem - just use StaticArrays or Tuples instead, and functional approaches like map/reduce rather than loops that write to arrays.

@rafaqz
Copy link
Member

rafaqz commented Mar 20, 2024

This is also bad, what is the reason to do this? The neighborhood is alread a StaticArrays StaticArray you can use math on directly - 10x faster than using an Array. neighbors(hood) gives you an actual SVector if thats simpler.

nbrvalues = [neighborhood[1],neighborhood[2],neighborhood[3],neighborhood[4],neighborhood[5],neighborhood[6],neighborhood[7],neighborhood[8]]

@rafaqz
Copy link
Member

rafaqz commented Mar 20, 2024

This fill also allocates. Why not just broadcast the addition?

nv=p*nbrvalues./(p*nbrvalues+fill(cell*(1-p),8)) 
nv=p.*neighborhood./(p.*neighborhood.+cell.*(1-p) 

Also broadcast everything else so it fuses.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants