CUGAN models doesnt work on RTX 3090

[Juliu5LT]

using TensorRT every CUGAN model gives CUDA out of memory on my RTX 3090 24 GB VRAM, Direct or NCNN works but lots of encoding errors in the video

[n1c3c0d3]

I have a 3090 as well, used TensorRT with mkv and got audio only. If I can get output from Direct or NCNN then maybe TensorRT has an issue.

[Juliu5LT]

TensorRT gives me CUDA out of memory error every time, Direct and NCNN i get output but very buggy lots of green screen artifacts etc etc

[n1c3c0d3]

So the one setting I wasn't looking out for was "allow videos in output path to be overwritten"

Disabling this at least got me as far as actually seeing the conversion in real time on console output.

I'm going to see if I hit the CUDA memory limit but so far that's not happening as it's only using less than 4gb of my vram.

Are you using NVENC?

[Juliu5LT]

Tried HEVC , x265 , x264 all 3 of em have artifacts for me using cugan...

[n1c3c0d3]

I'm wondering about your cpu as well, it's possible there is a bottleneck causing additional processing to go towards your vram. I have 12 cores and the vram usage is pretty low.

[Juliu5LT]

i9-9900K , 64 GB RAM

[n1c3c0d3]

Maybe share the output in a txt file and look for any errors in the console

[n1c3c0d3]

Forgot to ask, are you using compact or ultracompact?

[Kuphi]

I tried Real-CUGAN, which is not working on an RTX4080. If anyone has a good onnx model and/or TensorRT settings that work for Real-CUGAN, I would love to give it a try.