Helm/Upstream Charts/kserve

Chart name: kserve-resources, kserve-crd-minimal

Chart version: v0.17.0

Phab task:

• https://phabricator.wikimedia.org/T416580
• https://phabricator.wikimedia.org/T419722

Reviewers: Dawid Pogorzelski, Luca Toscano

Link to repo: oci://ghcr.io/kserve/charts/kserve , oci://ghcr.io/kserve/charts/kserve-crd

Link to source code: https://github.com/kserve/kserve/tree/master/charts

Topics:

• machine learning
• inference

Decision: pass

Reasoning:

This effort is to align current upstream kserve charts with internal modified copies to benefit from latest additions and updates from upstream.

Upstream modifications are outlined in detail in the following document.

Particular emphasis has to be put on the following part:

## ClusterRoleBinding Security Consideration

The upstream KServe chart uses a ClusterRoleBinding that grants the KServe controller permissions across all namespaces. This is a known security trade-off:

**Security concern:**

- The ClusterRoleBinding is namespace-agnostic, granting the KServe controller cluster-wide permissions
- Wikimedia has non-KServe namespaces and may add more in the future
- Ideally, this should be reworked to use RoleBindings per namespace for least-privilege

**Why we keep the ClusterRoleBinding:**

Namespace-scoped RoleBindings were evaluated but present a significant operational problem: when you call `helmfile destroy` on a specific InferenceService namespace (e.g., `edit-check`), it destroys the Role that has permissions to clean up resources in that namespace. This creates a catch-22 where you can no longer destroy InferenceServices cleanly.

**Rejected workarounds:**

1. **Not deleting roles/bindings on destroy** — leaves orphan resources requiring manual cleanup
2. **Pre-delete hooks on all ml-services** — would wire ml-services into chart assumptions and add complexity

**Future direction:**

We could explore an automatic mechanism to add RoleBindings when new InferenceService namespaces are created, or follow up with upstream to incorporate a namespace-scoped RBAC option.

Review

The security issue has been present for quite a while in deployment-charts, since the kserve chart was imported as yaml-single-file-config years ago. The Machine Learning team is going to work on improving it over time, but at the moment (April 2026) I didn't seem fair to stop the introduction of this new chart since it is a huge improvement compared to the previous status quo.

The new charts are worked on directly by upstream, there is a good community around them and upstream recommends this way over the previous one nowadays.