Clarify onchange: NQE compatible? use Performance Timeline instead? about netinfo HOT 15 CLOSED

I think it's not just a matter of clarifying onchange but clarifying (or replacing) downlinkMax as well.
Also, I think we should distinguish high level features here from low level ones.

The high level features (downlinkMax and onchange at the moment) should be designed to expose a single, useful value that would cover many of the use cases outlined.

The low level features (e.g. a Performance Timeline based raw network data which may include bytes downloaded per time period, as well as RTTs witnessed during that time) would be destined to explain the high level features as well as enable developers to experiment with their own network estimation algorithms, in the hope that better ones would evolve.

In the spirit of the Extensible Web, I don't think one should come at the expense of the other.

To sum up my position regarding the high level features from the blink-dev thread:

• downlinkMax should be replaced by (bikeshedding aside) expectedDownlink. A value which is defined as the expected bandwidth can then be refined with network estimation implementations.
• The expectedDownlink value should be clamped in order to prevent authors from relying on internal implementation details and make sure we can experiment with network estimation in the future and prevent variance and racy app logic.
  • I think some sort of "logarithmic clamping" can be useful, where the value is clamped to 10KB below 100KB, 100KB below 1000KB, and to 1000KB above that. (or any other mechanism that would have a similar effect)
• onchange can then represent actual significant bandwidth changes, when these changes are greater than the clamping value.

/cc @nrooney for opinions on the feasibility of an "expected bandwidth" set of values we could use until NQE is operational.

from netinfo.

downlinkMax should be replaced by (bikeshedding aside) expectedDownlink. A value which is defined as the expected bandwidth can then be refined with network estimation implementations.

Is the argument here that downlinkMax is defined as a ceiling, whereas in ~NQE world the value is the expected rate.. but may be exceeded during transfer?

The expectedDownlink value should be clamped in order to prevent authors from relying on internal implementation details and make sure we can experiment with network estimation in the future and prevent variance and racy app logic.

I don't see how clamping prevents us from experimenting with network estimation or reduces variance / racy app logic. The developer writes the same check (if (connection.downlinkMax < X) {...}) regardless, and that's the reason why we defined it as a double instead of an enum. That's not to say that clamping is a bad idea, but I don't think these are compelling reasons to put it in place.

onchange can then represent actual significant bandwidth changes, when these changes are greater than the clamping value.

"Significant" is not well defined here. I'd argue that change in underlying connection type is very significant as it also means that the user's last-mile may have changed drastically - e.g. different routing latency in carrier network, different throughput, etc. We shouldn't hide this from applications.

Note that onchange is not restricted to connection type changes. It's perfectly reasonable (and allowed by current definition) to trigger it whenever connection type changes and when an estimate is refined within the same network type -- e.g. "on HSPA" -> "switched to LTE" -> "estimated throughput 5mbps on LTE" is a reasonable chain of "significant" onchange events; without NQE, or when NQE doesn't know anything about the particular network, you'll get HSPA -> LTE.. you can't escape this base case.

The high level features (downlinkMax and onchange at the moment) should be designed to expose a single, useful value that would cover many of the use cases outlined.

I think this is counter to what you actually want. This is a complicated space with many variables and experience shows that attempting to boil it down to a single value ends up doing more harm than good. We need to provide the low-level building blocks that allow app developers to build their own frameworks on top.

The low level features (e.g. a Performance Timeline based raw network data which may include bytes downloaded per time period, as well as RTTs witnessed during that time) would be destined to explain the high level features as well as enable developers to experiment with their own network estimation algorithms, in the hope that better ones would evolve.

Number of downloaded bytes is already covered by Resource Timing. It's trivial to build an aggregate counter on top of that, and apps do need more than that - e.g. aggregate by origin, measure download of a particular resource, etc. As such, I'd say this out of scope for NetInfo.

Same goes for RTT's. Resource Timing allows you to get handshake times for new connections and once you have transferSize + duration you can do the basic math to get a moving average of your RTT's.

In terms of exposing NetInfo data via Performance Timeline...

Hmm, I don't really see the functional difference between, ~connection.on vs Performance Timeline delivery: both emit events, and these events can carry arbitrary attributes (Mbps estimate, RTT, confidence, etc)...?

To be clear, I'm not opposed to surfacing Mbps estimates via Perf Timeline, but I'm trying to understand the motivating use cases for requiring it to be so.. Am I missing anything here?

from netinfo.

Is the argument here that downlinkMax is defined as a ceiling, whereas in ~NQE world the value is the expected rate.. but may be exceeded during transfer?

The argument is that since downlinkMax is an unattainable and unrealistic maximum value, switching from it to an NQE value (which would be significantly smaller) would degrade the user's experience, since the values we provide would decrease significantly for similar network conditions. e.g. the switch to NQE would cause a UMTS connection which previously saw a 2MB downlinkMax, to possibly get dropped to ~300kbps, which will cause sites to recognize it as Edge, and degrade the user experience, even if an effective BW of 300kbps is enough to deliver the higher end experience.

I don't see how clamping prevents us from experimenting with network estimation or reduces variance / racy app logic. The developer writes the same check (if (connection.downlinkMax < X) {...}) regardless, and that's the reason why we defined it as a double instead of an enum. That's not to say that clamping is a bad idea, but I don't think these are compelling reasons to put it in place.

Clamping would prevent (or discourage) sites from concluding network type from downlinkMax values. (so discourage if (connection.downlinkMax == 0.384) isEdge = true; kind of logic, which would prevent us from providing anything other than 384 for these networks or content would break).

It would also prevent the related Client-Hint from being overly variant, as well as potentially decrease the privacy risk of the feature.

"Significant" is not well defined here. I'd argue that change in underlying connection type is very significant as it also means that the user's last-mile may have changed drastically - e.g. different routing latency in carrier network, different throughput, etc. We shouldn't hide this from applications.

Note that onchange is not restricted to connection type changes. It's perfectly reasonable (and allowed by current definition) to trigger it whenever connection type changes and when an estimate is refined within the same network type -- e.g. "on HSPA" -> "switched to LTE" -> "estimated throughput 5mbps on LTE" is a reasonable chain of "significant" onchange events; without NQE, or when NQE doesn't know anything about the particular network, you'll get HSPA -> LTE.. you can't escape this base case.

I don't disagree here. The question that pops up though, is what do you do when your NQE estimate changed from 5Mbps to 5.004Mbps? With clamping in place, I suggest that onchange will only fire when an observable change happened (so when estimate changed from 5Mbps to 6Mbps, or whatever clamping values we'd find reasonable).

To be clear, I'm not opposed to surfacing Mbps estimates via Perf Timeline, but I'm trying to understand the motivating use cases for requiring it to be so

I must have done a poor job explaining myself, but I don't think that the bandwidth estimates should be exposed using perf timeline. When I talked about exposing perf timeline based network info, I was talking about exposing the building blocks that would enable an implementation of bandwidth estimation in user space, so the raw network data that the browser is using for NQE. Personally, I doubt that the level of accuracy that Resource Timing provides in that space is enough to create a better NQE in user space, but OTOH, exposing raw network data may bring along its own bag of worms when it comes to privacy.

In any case, I agree that this is out of scope for net info.

from netinfo.

The argument is that since downlinkMax is an unattainable and unrealistic maximum value, switching from it to an NQE value (which would be significantly smaller) would degrade the user's experience, since the values we provide would decrease significantly for similar network conditions. e.g. the switch to NQE would cause a UMTS connection which previously saw a 2MB downlinkMax, to possibly get dropped to ~300kbps, which will cause sites to recognize it as Edge, and degrade the user experience, even if an effective BW of 300kbps is enough to deliver the higher end experience.

That's a good argument for why developers need to know both. E.g. you may be on LTE (with low latency) but be rate-limited due to exceeded BW cap on your plan, which is a very different scenario from being on a 2G connection with same 'effective BW'.

FWIW, had an offline chat with @jkarlin yesterday and he convinced me that we should separate maxDownlink from NQE because they have different semantics:

• maxDownlink is the ceiling of your first hop, which is based on connection type+subtype and local network weather - e.g. Android's wifi API will provide the max data rate based on current signal quality, some mobile chipsets are also capable of surfacing similar information.
• downlink can be the NQE value, when/if we get to implementing it.
• onchange should be triggered whenever type, maxDownlink, or downlink change. The developer subscribes to a single notification, and each event carries all three values at the point when event is generated.

Clamping would prevent (or discourage) sites from concluding network type from downlinkMax values. (so discourage if (connection.downlinkMax == 0.384) isEdge = true; kind of logic, which would prevent us from providing anything other than 384 for these networks or content would break).

It doesn't. We'll round it up 400kbps and same bad developer will do the same bad pattern. The reason we are providing a double instead of an enum is to enable developers to do range comparisons.

It would also prevent the related Client-Hint from being overly variant, as well as potentially decrease the privacy risk of the feature.

Caching: Key :). Privacy: bucketing will help a bit, but not much, we still need to think through permissions here - let's take this up in https://github.com/w3c/netinfo/issues/26.

I don't disagree here. The question that pops up though, is what do you do when your NQE estimate changed from 5Mbps to 5.004Mbps? With clamping in place, I suggest that onchange will only fire when an observable change happened (so when estimate changed from 5Mbps to 6Mbps, or whatever clamping values we'd find reasonable).

Yes, we'd probably have to put some guidance in place for this. That said, as long as we keep NQE as a separate signal from maxDownlink I think this is something we can defer until the time we're ready to expose it.

from netinfo.

maxDownlink is the ceiling of your first hop, which is based on connection type+subtype and local network weather - e.g. Android's wifi API will provide the max data rate based on current signal quality, some mobile chipsets are also capable of surfacing similar information.
downlink can be the NQE value, when/if we get to implementing it.
onchange should be triggered whenever type, maxDownlink, or downlink change. The developer subscribes to a single notification, and each event carries all three values at the point when event is generated.

Great! I think onchange may not be a good fit to cover changes in all three attributes, but we can look into that when NQE is more mature, and it's time to actually expose downlink.

I still think that most use-cases would be covered by downlink, but since some would be covered by downlinkMax, which is easy to expose, and there are good use-cases to exposing both, I'm now convinced that we should (privacy permitting).

Still, we have to be vigilant and make sure that our messaging to developers reflects downlinkMax correctly, and make it clear that a high downlinkMax value doesn't mean bandwidth is abundant, cost is low or other false assumptions.

from netinfo.

Not saying we should, but to force the developer to deal with an unknown downlinkMax we could make it nullable

from netinfo.

Great, I think we have high-level consensus here.

Took a run at clarifying wording around when the change event should be triggered: w3c/netinfo@d5a7101 - wdyt, is this a step in the right direction?

Re, nullable: I'm on the fence and could be convinced either way.. I see the argument for setting it to null when connection type is 'unknown', but it's also a bit annoying since now every lookup needs to have another conditional.. and I'm guessing developers will simply end up doing max = connection.downlinkMax || 0/Infinity anyway?

from netinfo.

Took a run at clarifying wording around when the change event should be triggered: d5a7101 - wdyt, is this a step in the right direction?

LGTM

Re, nullable: I'm on the fence and could be convinced either way.. I see the argument for setting it to null when connection type is 'unknown', but it's also a bit annoying since now every lookup needs to have another conditional.. and I'm guessing developers will simply end up doing max = connection.downlinkMax || 0/Infinity anyway?

I don't think there's a practical difference between null and infinity for an unknown connection type, so having it nullable doesn't make much sense to me.

FWIW, the scenario that scares me is not the one of unknown connection type (where the result would be infinity), but one of known connection type where the theoretical first hop maximum is way higher than the effective bandwidth. We need to make it clear to developers what the downlinkMax semantics are, what they guaranty, and especially what they don't guaranty. But that's mostly education work rather than spec work.

from netinfo.

@jkarlin ptal: https://github.com/w3c/netinfo/pull/28

We need to make it clear to developers what the downlinkMax semantics are, what they guaranty, and especially what they don't guaranty. But that's mostly education work rather than spec work.

Anything in particular you have in mind, in terms of text, etc?

from netinfo.

I'd like to see a clear statement that downlinkMax is an upper bound on the next-hop bandwidth. The UA should make it as tight an upper bound as it can, but can default to +Infinity if the underlying connection technology is unknown.

from netinfo.

@jkarlin @yoavweiss ptal at https://github.com/w3c/netinfo/pull/29 - better?

from netinfo.

Hi all - I started reading this thread to make sure this post was applicable but between here and the google group you guys have written a fair amount! So, I'm going to dump this info here, then read the rest of the thread, and then look for more data. Yeah I know it's the reverse order, but I wouldn't be me without mixing it all up!

Anyway this data might not be exactly what Yoav was asking me for, but it does go by technical network release/air interface.

https://docs.google.com/spreadsheets/d/1aSjy0_nYgd0itQw38ZsdAX3oE2IeDpx8_e3YtAVq3m0/edit?usp=sharing

from netinfo.

Thanks Natasha! :)

Indeed this data is not exactly what I was looking for (I was more interested in expected bandwidth, for NQE seeding), but it does seem relevant to downlinkMax. (and slightly differs from https://w3c.github.io/netinfo/#max-downlink-table )

from netinfo.

@yoavweiss @nrooney that table is bootstrapped from https://github.com/w3c/netinfo/blob/gh-pages/downlink.json - see the ref's in there. If you see any issues with any of the values, please open a separate bug.

from netinfo.

Resolved via https://github.com/w3c/netinfo/pull/29.

from netinfo.