Setting inference seed is nowhere to be found in the doc

[londumas]

I have exported a SegNext-t to .engine, and I infer images with it. The issue is that this model is not deterministic since it uses matrix decomposition through random numbers. One simple way to do so in onnxruntime is trhough ort.set_seed(seed). However I can not find anywhere how to do the same for tensorrt in python. I do not find anything in the documentation about reproductibility.

[lix19937]

Try to use follow

import torch 
import numpy as np
import random 

def fix_seed(identical_seed : int = 1024):
  torch.manual_seed(identical_seed)
  torch.cuda.manual_seed(identical_seed)
  torch.cuda.manual_seed_all(identical_seed)
  np.random.seed(identical_seed)
  random.seed(identical_seed)
  torch.backends.cudnn.deterministic = True
  torch.backends.cudnn.benchmark = False

[londumas]

@lix19937 , thank you for your help. However this does not work, I tested it today.
However, it makes sense, since the code you send me, fixes seeds of packages that have nothing to do with tensorrt or pycuda.
I found a way to set the seed of a generator in pycuda, however this generator is not the same as the one used by tensorrt, so it does not help us here:

import pycuda.autoinit
from pycuda import curandom
import numpy as np
import pycuda

def seed_getter(N):
    array = np.array([42]*N).astype(np.int32)
    array = pycuda.gpuarray.to_gpu(array)
    return array

sampler = curandom.XORWOWRandomNumberGenerator(seed_getter=seed_getter)
print(sampler.gen_uniform(1000, np.float32).get()[0])

sampler = curandom.XORWOWRandomNumberGenerator(seed_getter=seed_getter)
print(sampler.gen_uniform(1000, np.float32).get()[0])

[lix19937]

@londumas Why do you not use the fixed inputs ?

[asfiyab-nvidia]

@londumas if your model contains random ops that require a seed input, TensorRT doesn't support specifying this input at the moment. We may add support in the future.

However, in the event that the input to your model is a randomly generated tensor, you can set the seed during input tensor initialization in PyTorch.