Currently our data plane config (proxy/v1/config/cfg.proto) and our control plane config (mixer/v1/config/cfg.proto) are quite different.
The mixer config is based on modeling user intent in a generic way, using aspects, which have a selector based on generic attribute model. This model is intended to be fully general, capable of modelling anything in the system. It was not meant as a "control plane only" configuration model. If we stray from this on any user-facing configuration models, I'd like to understand why and make sure we have very good reasons for doing so.
The data plane config is currently very specific to the problem it is trying to solve, with lots of hard-coded decisions made directly in the configuration model. This is troubling for a number of reasons, but the most basic is that it is inconsistent with the configuration philosophy of the rest of istio, where we specify intent in a uniform way. The benefit of this is that the proxy does not need to have a general purpose selection model, and can have potentially higher performance. I'm not sure about the performance aspect (we can "compile" selectors down into equivalent code if needed), but the need for a general selection model vs. a hand-coded selection based on a hand-coded api is certainly easier to get started with.
So I'm fine saying that we stick to a non-general-purpose model until we get the general purpose model in place, but we should agree on the end goal, that there is a uniform configuration model for the system as a whole, and that we can feed various artifacts (such as annotations on k8s native resources) into that generic model to produce a unified configuration object for any particular service.
So what would this look like in the short and medium term?
In the short term, we should be as close as we can to the end model while keeping the selector hand-coded. So that would look like this:
message ProxyConfig {
string subject = 1;
string revision = 2;
repeated RoutingRule rules = 2;
repeated UpstreamCluster clusters = 3;
}
message RoutingRule {
RoutingSelector selector = 1;
repeated RoutingAspect aspects = 2;
repeated RoutingRule rules = 3;
}
message RoutingSelector {
L4Selector l4_selector = 1;
HttpSelector http_selector = 2;
}
message L4Selector {
... // Existing L4MatchAttributes
}
message HttpSelector {
... // Existing HttpMatchCondition
}
message RoutingAspect {
string kind = 1;
map<string, string> inputs = 3;
google.protobuf.Struct params = 4;
}
Two of the RoutingAspects that we would build in would be:
message WeightedClusterAspect {
ClusterId dst_cluster = 1;
uint32 weight = 2;
}
message FaultInjectionAspect {
... // L4FaultInjection contents
}
This also gives us a nice place to add more routing aspects later, and lets partners plug in their own aspects once there is a plugin model.
In the long term, we'd drop the separate selectors and just have a string, and document what attributes are available for routing decisions. Then the proxy would have either its own implementation of the selector execution, or we'd limit the selectors available in the proxy to some simplistic functionality and make it a config validation error to use things that aren't supported.
There is a separate interesting discussion, which is how to model what we're currently putting in UpstreamCluster, but for general config modeling. How do we model reusable chunks of configuration, such as defining a quota group, an upstream cluster, etc? I'll open a separate issue about that too, let's keep this one about aligning the configuration models.