[P1] Look into potential conflict with Service Mesh integration

[heyselbi]

The Service Mesh integration with RHODS is in progress. This is a blocker until we have the service mesh integration with RHODS is fully completed.

---

As of July 14th, KServe on-boarding into ODH is, in practice, complete for the core components; i.e. the controller, agent, router and storage-initializer.

For ODH, Istio/Maistra Service Mesh was chosen as the networking layer. This means that Service Mesh is a pre-requisite for ODH-KServe.

In parallel, a tighter Service Mesh integration is being worked on. At the moment, that work is being done in feature branches and forks, until it is stable enough; e.g of some locations of the code:

• https://github.com/opendatahub-io/odh-manifests/tree/service-mesh-integration
• https://github.com/opendatahub-io/odh-model-controller/tree/service-mesh-integration
• https://github.com/bartoszmajsak/opendatahub-operator/tree/kf_ossm_plugin/pkg/kfapp/ossm
• https://github.com/maistra/odh-dashboard
• https://github.com/opendatahub-io/kubeflow/pulls?q=author%3Abartoszmajsak+
• etc.

At the time of writing, the most prominent point of conflict is authentication/authorization configurations. See related manifests here: (link). OAuth proxy is being replaced in favor of MeshSidecars+Authorino. The current setup is tailored to work well with notebooks and the odh-dashboard, but other components are not working good. For ModelMesh, there is a workaround (here and here).

The authorization/authentication configs are set on the Istio ingress-gateway. KServe also uses this gateway and doesn't expect such configs which effectively block infer requests.

[heyselbi]

@israel-hdez is this still an issue? If not, could you provide 1-2 sentence summary and close it out.

[israel-hdez]

It is still an issue. Although we can apply both KServe + ServiceMesh intergration work, configurations are not good and, AFAIK, infer requests become blocked.

[taneem-ibrahim]

@israel-hdez Please add the description for the issue

[israel-hdez]

@israel-hdez Please add the description for the issue

@taneem-ibrahim
I've updated the main comment with the details I know.

[israel-hdez]

Update:

The work done here is potentially a point of conflict: opendatahub-io/odh-model-controller#65.

I've already mentioned it to the people working on the Service Mesh integration to make them aware.

[israel-hdez]

This is a long-running task.

The SM integration work is being tracked in this ticket: opendatahub-io/opendatahub-community#97.

Since KServe was already adopted in ODH, perhaps it is better to address any conflict under the scope of the service mesh integration, which is being worked in feature branches. It looks like we simply need to provide guidance to enable the Service Mesh team for a smooth integration. Such conversation can happen in other channels.

So, I'm closing this ticket, because it is not actionable at the moment. We can later create new tickets with the specific actionable work, if any.