Unable to install TensorRT-LLM

[gowthamtupili]

Device Details:
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal

I am trying to build tensorRT-LLM for whisper, and I have followed the steps as mentioned in https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper#readme, but I am getting the following errors,
When I am using python version 3.8 and pip install -r requirements.txt command, it is giving me the following error
Collecting tensorrt_llm==0.17.0.dev2024121700 (from -r requirements.txt (line 1))
Using cached tensorrt_llm-0.17.0.dev2024121700.tar.gz (1.4 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [54 lines of output]
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp310-cp310-linux_x86_64.whl against tag cp310-cp310-linux_x86_64
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp312-cp312-linux_aarch64.whl against tag cp312-cp312-linux_aarch64
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp312-cp312-linux_x86_64.whl against tag cp312-cp312-linux_x86_64
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
return download_manual(wheel_directory, distribution, version, config)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 173, in download_manual
raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
Traceback (most recent call last):
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
return download_manual(wheel_directory, distribution, version, config)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 173, in download_manual
raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
RuntimeError: Didn't find wheel for tensorrt-llm 0.17.0.dev2024121700

  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
      whl_basename = backend.build_wheel(metadata_directory, config_settings)
    File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
      return download_wheel(pathlib.Path(wheel_directory), config_settings)
    File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 237, in download_wheel
      report_install_failure(distribution, version, config, exception_context)
    File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/error.py", line 67, in report_install_failure
      raise InstallFailedError(
  wheel_stub.error.InstallFailedError:
  *******************************************************************************
  
  The installation of tensorrt-llm for version 0.17.0.dev2024121700 failed.
  
  This is a special placeholder package which downloads a real wheel package
  from https://pypi.nvidia.com/. If https://pypi.nvidia.com/ is not reachable, we
  cannot download the real wheel file to install.
  
  You might try installing this package via
  ```
  $ pip install --extra-index-url https://pypi.nvidia.com/ tensorrt-llm
  ```
  
  Here is some debug information about your platform to include in any bug
  report:
  
  Python Version: CPython 3.8.3
  Operating System: Linux 5.15.0-125-generic
  CPU Architecture: x86_64
  Driver Version: 560.28
  CUDA Version: 12.6
  
  *******************************************************************************
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

However, when I change the python version from 3.8 to 3.10, and run the same command

/home/mcw/anaconda3/envs/py310/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to `opal_list_t_class'
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
error: Cannot link MPI programs. Check your configuration!!!

ERROR: Failed building wheel for mpi4py

My understanding is that
Problem:
mpi4py only works with python version 3.8 or else it will get error
tensorrt-llm only works with python version 3.10

[zhangts20]

You can try adding the directory containing libopen-rte.so.40 to the LD_LIBRARY_PATH.