The endpoint url for vLLM inference servers that are deployed using vLLM Serving Runtime are insecure. To configure the KServe to use signed certificates to generate secure routes
-
Copy secrets letsencrypt-production-key and aws-route53-credentials from openshift-ingress namespace into istio-system namespace.
-
Create an Issuer for the istio-system namespace i. Navigate to Administration → Custom Resource Definitions → Issuer ii. Create an Issuer based upon an existing Issuer in the istio-system.
apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: letsencrypt-production-dns01 namespace: istio-system labels: nerc.mghpcc.org/kustomized: 'true' spec: acme: email: help@nerc.mghpcc.org privateKeySecretRef: name: letsencrypt-production-key server: 'https://acme-v02.api.letsencrypt.org/directory' solvers: - dns01: cnameStrategy: Follow route53: accessKeyIDSecretRef: key: AWS_ACCESS_KEY_ID name: aws-route53-credentials region: us-east-1 secretAccessKeySecretRef: key: AWS_SECRET_ACCESS_KEY name: aws-route53-credentials selector: dnsZones: - ocp-beta-test.nerc.mghpcc.org
-
Navigate to Administration → Custom Resource Definitions → Issuer
-
Create a Certificate rhoai-letscrypt-cert based upon existing certificate default-ingress-certificate
apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: rhoai-letscrypt-cert namespace: istio-system spec: dnsNames: - '*.apps.ocp-beta-test.nerc.mghpcc.org' duration: 2160h0m0s issuerRef: kind: Issuer name: letsencrypt-production-dns01 renewBefore: 360h0m0s secretName: rhoai-letscrypt-cert
-
A secret rhoai-letscrypt-cert should be created in the istio-system namespace
-
Navigate to Operators → Installed Operators → Red Hat OpenShift AI → Data Science Cluster
-
Click on default-dsc → YAML
-
For the kserve component update the certificate section to use the provided certificate. Secret name should be the same as you configured while creating the certificate.
spec: components: codeflare: managementState: Managed kserve: managementState: Managed serving: ingressGateway: certificate: secretName: rhoai-letscrypt-cert type: Provided managementState: Managed name: knative-serving
The rhoai-operator does not reconcile the Gateway correctly. To verify if the operator has reconciled correctly
-
Navigate to the Administration → Custom Resource Definitions → Gateway
-
Click on Gateway → Instances
-
Click on knative-ingress-gateway → YAML. Verify that the tls section is present.
apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: knative-ingress-gateway namespace: knative-serving spec: selector: knative: ingressgateway servers: - hosts: - '*.apps.ocp-beta-test.nerc.mghpcc.org' port: name: https number: 443 protocol: HTTPS tls: credentialName: rhoai-letscrypt-cert mode: SIMPLE
-
If the tls section is not present, force the operator to reconcile manually:
i. Navigate to Operators → Installed Operators → Red Hat OpenShift AI → Feature Tracker ii. Search for Gateways and delete the Feature Tracker. It will be recreated.
iii. Navigate to Workload → Pods in the redhat-ods-operator
iv. Delete the rhodes-operator pod. It will be recreated. v. Verify the Gateway Instance as described above.