ray-project / kuberay-helm Goto Github PK
View Code? Open in Web Editor NEWHelm charts for the KubeRay project
Helm charts for the KubeRay project
When installing ray operator with the following settings
singleNamespaceInstall: true
rbacEnable: true
crNamespacedRbacEnable: true
watchNamespace:
- xxx
we get the error
E1122 16:36:19.205328 1 reflector.go:138] go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1.Job: failed to list *v1.Job: jobs.batch is forbidden: User "system:serviceaccount:default:kuberay-operator" cannot list resource "jobs" in API group "batch" in the namespace "xxx"
I believe this is due to missing rules from
https://github.com/ray-project/kuberay-helm/blob/main/helm-chart/kuberay-operator/templates/multiple_namespaces_role.yaml
The batch.jobs
RBAC permission is present in the cluster role equivalent here
kuberay-helm/helm-chart/kuberay-operator/templates/role.yaml
Lines 10 to 21 in 07463a1
I'm playing around with the combination of Kuberay and Kueue for quota management. In order to have Kueue manage the resource quotas for RayCluster
s, I need to put an additional label on the CR (not the head
/worker
pods; see the linked docs).
The current Helm chart unfortunately does not expose a way to do this easily, so I have to resort to helm template
and manually add the label in the rendered manifest (I can't do kubectl label
, either, since the admission controller apparently does not allow modifying the labels on an existing RayCluster
CR).
Would it be possible to add a way in the Helm chart to pass extra labels to be applied to the RayCluster
CR via values.yaml
? I can provide a PR, if you are interested.
kuberay-helm/helm-chart/kuberay-operator/values.yaml
Lines 76 to 79 in 07463a1
singleNamespaceInstall
.
If we install apiserver, operator in namespace X
and operator with watchNamespac [X, Y]
, trying to manage CRD in Y
, the setup leads to
2023/11/22 16:56:17 could not list clusters rpc error: code = Unknown desc = List clusters failed.: List RayCluster failed in tsray: rayclusters.ray.io is forbidden: User "system:serviceaccount:default:kuberay-apiserver" cannot list resource "rayclusters" in API group "ray.io" in the namespace "Y"
Should we consider also installing the RBAC for CRD in kuberay-apiserver
to a list of watchNamespace
like kuberay-operator
?
I'm happy to provide that PR if maintainers believe this is a reasonable request
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.