NVIDIA Riva for ASR and TTS on OpenShift

Requirements

For both physical and virtual GPUs you need:

• Access to a plan with on of the supported GPU models on Equinix Metal
• Access to NGC Catalog
• NGC CLI tool installed and configured
• Helm 3 installed

For physical GPU

• Requirements as described in Red Hat OpenShift on Equinix Metal with GPU.

For vGPU

• Requirements as described in Deploying an OpenShift Cluster with vGPU on VMware vSphere.

Preparation

Populate required and optional variables as described in the GPU playbook's documentation. If needed, populate also the additional variables fot vGPU.

In addition, add variables for pulling NVIDIA images from NGC.

ngc_api_key: <NGC API key>
ngc_email: <NGC email>

Download Riva artifacts and a playbook for installing an SNO cluster on Equinix by running:

for physical GPU

ansible-playbook init-gpu.yml -e "@path/to/vars.yml"

for vGPU

ansible-playbook init-vgpu.yml -e "@path/to/vars.yml"

Running

Riva API Server

Run the playbook

Physical GPU

ansible-playbook riva-gpu.yml -e "@path/to/vars.yml"

vGPU

ansible-playbook riva-vgpu.yml -e "@path/to/vars.yml"

In order to destroy the setup, run — depending on the GPU type (physical/virtual) — either

ansible-playbook gpu-openshift-playbook/sno-cleanup.yml -e "@path/to/vars.yml"

or

ansible-playbook vgpu-openshift-playbook/destroy.yml -e "@path/to/vars.yml"

In order to delete just a Riva deployment, run

export KUBECONFIG=...
helm delete -n riva riva-api

Riva Client

Install the Riva API wheel that can be found under the downloaded quick start directory (e.g. riva_quickstart_v2.2.1).

Then for offline speech recognition run

python riva_quickstart_v2.2.1/examples/transcribe_file_offline.py --server <cluster_node>:<node_port> --audio-file <audio_sample.wav>

e.g.

python riva_quickstart_v2.2.1/examples/transcribe_file_offline.py --server 147.28.142.251:32222 --audio-file Sports.wav

For real-time speech recognition using a microphone, first run

python riva_quickstart_v2.2.1/examples/transcribe_mic.py --list-devices

then select your input device ID and run

python riva_quickstart_v2.2.1/examples/transcribe_mic.py --server <cluster_node>:<node_port> --input-device <input_device>

WARNING: You will need to install dependencies such as PyAudio. The exact installation procedure depends on the system you are running on. Keep in mind that PyAudio is currently broken in Python 3.10. Use an earlier version of Python, for instance by creating a virtual environment (venv). You may also need to adjust the sample rate of your input device.

See the riva_quickstart_v2.2.1/examples directory for other quick start Python clients.

Additional Resources

• Riva Helm chart documentation
• Riva Quick Start Guide
• Riva ASR Overview
• Riva Tutorials
• Pre-trained Riva models on NGC
• Some 3rd party audio samples here, here and here