Out of memory error

[AgeOfAlgorithms]

Hello! I'm a big fan of this repo. I'm using a rx 6700xt GPU with 12GB vram with rocm on Ubuntu. I get an out of memory error.

Error occurred when executing OOTDGenerate:

HIP out of memory. Tried to allocate 4.50 GiB. GPU 0 has a total capacty of 11.98 GiB of which 1.99 GiB is free. Of the allocated memory 9.59 GiB is allocated by PyTorch, and 47.22 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_HIP_ALLOC_CONF

  File "/home/sean/ComfyUI/execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/__init__.py", line 170, in generate
    images = pipe(
             ^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/inference_ootd.py", line 152, in __call__
    images = self.pipe(
             ^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/pipelines_ootd/pipeline_ootd.py", line 354, in __call__
    _, spatial_attn_outputs = self.unet_garm(
                              ^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/pipelines_ootd/unet_garm_2d_condition.py", line 1079, in forward
    sample, res_samples, spatial_attn_inputs = downsample_block(
                                               ^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/pipelines_ootd/unet_garm_2d_blocks.py", line 1172, in forward
    hidden_states, spatial_attn_inputs = attn(
                                         ^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/pipelines_ootd/transformer_garm_2d.py", line 381, in forward
    hidden_states, spatial_attn_inputs = block(
                                         ^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/ComfyUI/custom_nodes/ComfyUI-OOTDiffusion/pipelines_ootd/attention_garm.py", line 264, in forward
    attn_output = self.attn1(
                  ^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/diffusers/models/attention_processor.py", line 522, in forward
    return self.processor(
           ^^^^^^^^^^^^^^^
  File "/home/sean/anaconda3/envs/ootd/lib/python3.11/site-packages/diffusers/models/attention_processor.py", line 1231, in __call__
    hidden_states = F.scaled_dot_product_attention(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Is this node impossible to run on my GPU?

[iyume]

Yes. I used to handle this on 12GB GPU.

I made a new branch here, can you try it?

git fetch origin 12gb
git switch 12gb

I am not planning to merge this to main branch because it is slower about three seconds.

[AgeOfAlgorithms]

Yes. I used to handle this on 12GB GPU.

I made a new branch here, can you try it?

git fetch origin 12gb
git switch 12gb

I am not planning to merge this to main branch because it is slower about three seconds.

I switched to the new branch and tried again, and I get the same error

[AgeOfAlgorithms]

sorry, I misclicked on close issue button

[AgeOfAlgorithms]

@iyume Here are some additional info from my system:
ROCm version: 5.6.1.50601
Ubuntu version: 22.04.4 LTS
Python version: 3.11.8
Pytorch version: 2.2.0.dev20231010+rocm5.6

I've seen other people using ROCm get this same error on different nodes and workflows,
like this reddit post for example: https://www.reddit.com/r/comfyui/comments/1bo3940/img2vid_lcm_help_why_does_it_allocate_30gb/

[yurayko]

12gb branch not worked on RX6800 with 16gb too.
HIP out of memory. Tried to allocate 18.00 GiB. GPU
rocm 6.1