Promote canary to default without creating new deployment about kserve HOT 11 CLOSED

I'm a little wary of "upon promotion". I don't think our code should have any concept of promotion, as it introduces statefulness of the cluster. Perhaps this is acceptable in the optimization case, but it still smells weird to me.

I do agree that we don't want to be eating the cost of flipping all the resources around.

from kserve.

I might be more comfortable if we tracked a hash of the kfspec. If a knconfiguration owned by our KFService is already serving that spec, we can re-use it. Perhaps we can mark the configuration with a hash?

This optimizes both the case where canary == default and the case of flipping canary to default and deleting canary. We just need to make sure that both canary & default are not using a configuration before cleaning one up. Perhaps we can track this with owner references on the configuration?

I don't think this is too different from what you're suggesting. What do you think?

from kserve.

Seems like optimization that we can punt for now?

from kserve.

Going back to my earlier philosophy which I commented - why is the focus here on enhancing knative functionality? If knative is going to serve 80% of world's microservices (hypothetically), what is it in model-serving which is so fundamentally different? If we are solving Knative's deficiencies, lets target that community and contribute there. The goal here should be to NOT deviate from Knative's defaults, so that tools and ecosystem which emerge in the community can be used (e.g. monitoring etc..).

from kserve.

If there are domain specific usecase in Model serving which require a fundamentally different approach than knative, that makes sense. To me a lot fo discussions here seem to be around general enhancements on canary deployments, routing etc.... which ideally should be taken up on Knative side, rather than implementing this here

from kserve.

+1 @animeshsingh. I do think that this is an artifact of how we're treating Kn Configurations, and it may not necessarily be a common pattern for them to optimize around, but either they should have a better way for us to use Knative, or they should implement this optimization.

I think the principle you allude to of "KFServing solves ML problems, Knative solves deployment problems" is a good guiding principle for us as we prioritize our efforts.

from kserve.

@animeshsingh definitely agree with your points. We have discussed the solution with knative team and knative serving is designed in a flexible way that can enable user for more specific pattern with the component they provided(here we use two knative configurations instead of one service), but it may worth to discuss with them if our pattern is common enough which can be pushed down to knative.

from kserve.

/area control-plane
/priority p2

from kserve.

"here we use two knative configurations instead of one service" - where can find the rationale for this? we have many knative contributors in IBM, and i would want to pass it by them to get opinion as well

from kserve.

/area performance

from kserve.

/kind feature

from kserve.