High resolution timing for events about dom HOT 70 CLOSED

• Bugs: A Webkit bug is already filed. We just need to file one against Edge.
• We wrote a some tests when we landed Blink implementation. They are written in web-platform-test style and I am pretty sure I can upstream at least two without any change.
• Happy to send a pull-request against the spec if no one else does it in the meantime.

from dom.

After almost three years of testing with Firefox Nightly and Dev Edition users, Mozilla plans to ship high-resolution Event.timeStamp in Firefox 54 (2017-06-13). See @birtles' Intent to Ship announcement.

from dom.

@birtles, what is the timeline for this?

from dom.

I expect to finish converting our Linux UI event timestamps over on non-release channels this quarter. Still need to find someone to do the Mac work so we can switch this on for release channels.

from dom.

We (@majido) are working on this in blink too. There are two completely different things coupled together here:

1. Use logical event time instead of strict "event creation time".
2. Use a monotonic/high-precision time (DOMHighRestTimestamp) instead of an epoch time.

It's 1 that's most important IMHO, giving us two key properties:

• Ability to accurately compute pointer velocity
• A basic way to measure input latency (very important for tracking improvements to some class of perf problems IMHO).
  But it's also potentially hardest to spec. Eg. does the spec currently require that events are delivered in increasing timestamp order? If we rely on OS/device timestamps then it's possible, for example, that a touchmove may be dispatched after a mousemove with a later timestamp.

WebKit has done 1 without 2 for a couple years now, but it's a kludge. Given the lack of use cases for an epoch-based event timestamp and the existing lack of interoperability here, I agree we should also try to change the semantics of timestamp.

from dom.

This might not be web-compatible: https://bugzilla.mozilla.org/show_bug.cgi?id=1231619#c3. But perhaps browsers are willing to push it through anyway...

from dom.

Yep, @majido says the compat impact seems manageable, so we're still pushing ahead with shipping in Chrome 49 (currently in beta, set to go stable in early March). If we have non-trivial compat issues we'll revert and revisit adding a new property instead.

from dom.

Some fallout of the change: https://code.google.com/p/chromium/issues/detail?id=574514

from dom.

I find it quite surprising that this is considered an acceptable web-compatible change, for the reasons that @chancancode discussed on the Bugzilla thread.

The jQuery API docs and MDN both define event.timeStamp as "The difference in milliseconds between the time the browser created the event and January 1, 1970". Incorrect or not, the practical effect of this is that lots and lots of code exists in the wild that assumes that new Date() and event.timeStamp can be compared to produce a relative amount of milliseconds.

Of course, new Date() and the old event.timeStamp are not very reliable as they are not monotonic (I'm very familar with these problems as the author of the RFCs that added Duration and timestamp functionality to Rust)

That said, I am shocked that decoupling the meaning of event.timeStamp from new Date() is considered even remotely web compatible. Immediate fallout included a demo @chancancode and I happened to be working on for the Ember rendering engine, Angular animation code, and the MSE test suite reported by @foolip.

A more conservative solution to the problem would be to introduce a new attribute (event.time as a nice shorter name? event.performance.timeStamp for symmetry with window.performance.*?).

Given the kinds of things that have blocked shipping recently for web-compat reasons, and the number of affected properties, I just don't understand how we're full steam ahead here. (In the linked case, the problematic feature had already shipped in IE11 and Edge years earlier, which made the number of affected properties quite small.)

To quote @chancancode:

By the way, if you are thinking that code that compares against wall-clock time is fundamentally "incorrect", I totally agree with you, and I am personally looking forward to using this feature. However, correctness is irrelevant here because event timestamps have always been tied to wall-clock time, and there are no better way to do this today. The fact is it basically work "good enough" for the majority cases that people have been able to build useful things on top of it today, and we should be careful not to break them. (For example, popular libraries like underscore uses Date.now for debouncing/throttling – while "incorrect", it obviously works "good enough" to be useful in the wild.)

I would quote him in full, but I'd recommend just reading the linked Bugzilla thread.

from dom.

"Lots and lots of sites" has not been our experience. But as with anything compat, data trumps anecdote here. @majido, can you share your analysis from httparchive data? Anyone else have more examples of specific sites impacted? If we find a pattern that, say, we find used on >0.01% of the top sites in httparchive then we should definitely reconsider.

FWIW I argued ages ago for a new Event systemTime property, and we went around-and-around for years and couldn't get it spec'd anywhere due to constant debate about whether it was better to add a new property (if so which spec) or fix the old one. Firefox never implemented the UNIX-epoch version, so felt changing the timebase was already largely web compatible and understandably didn't want to regress to using wall-times in a property that was already (rightly) monotonic for them.

So attempting to ship a breaking change in Chrome was our compromise. So far it's hit Chrome beta with millions of users and very little fallout, so we have no reason to pull the plug yet. We're still willing for new data to change our mind, but the time for arguing-absent-hard-data (after more than 3 years of such arguing blocking solving the use cases here) is done IMHO.

from dom.

So attempting to ship a breaking change in Chrome was our compromise. So far it's hit Chrome beta with millions of users and very little fallout, so we have no reason to pull the plug yet.

The problem with timing bugs is that they don't necessarily manifest as hard errors, by definition. Given how flaky some websites can be already, it may take some time before people report the bug, even to the original author, and before that bug gets reported upstream.

In this case, we rapidly discovered a (rather subtle) bug in a very popular animation library (Angular) that is easy to reproduce. How is that not sufficient?

(the fact that Angular has addressed the issue in 1.5 doesn't really do much -- some existing content will upgrade, but plenty won't)

from dom.

If we find a pattern that, say, we find used on >0.01% of the top sites in httparchive then we should definitely reconsider.

Is there a way we can test uses of:

• e.timeStamp - anyDate
• anyDate - e.timeStamp
• var d = e.timeStamp, later: anyDate - d or d - anyDate

I'd be very very surprised if those patterns did not impact a non-trivial number of the top sites.

from dom.

The problem with timing bugs is that they don't necessarily manifest as hard errors, by definition. Given how flaky some websites can be already, it may take some time before people report the bug, even to the original author, and before that bug gets reported upstream.

Yeah I agree that's cause for extra concern here. Also there may be non-visual cases where the breakage is hidden from us, eg. where the client sends the timestamp to the server for some analytics.

In this case, we rapidly discovered a (rather subtle) bug in a very popular animation library (Angular) that is easy to reproduce. How is that not sufficient?

We believe the impact in practice is small (due to low usage of the particular affected feature from our httparchive digging and as a result of being cosmetic). @majido can provide the concrete data from his analysis.

Is there a way we can test uses of: ...

Yes, the first two are, I believe, the analysis @majido has done on httparchive data for the top 470k websites. I'll let him provide details. Such code has always been broken on Firefox (except for the specific event that Angular was using, due to a bug in Firefox where it's timestamp was always 0), so it does make sense that developers haven't been relying on this.

I should add that unfortunately I'm out on vacation without internet for the next 1.5 weeks. Please continue discussing the tradeoff with Majid - he owns this feature on blink. If you like you're welcome to follow-up on the Intent to ship thread, it's possible other blink API owners with disagree with our judgement here and request the change be un-shipped.

from dom.

BTW, some devrel outreach around this here

from dom.

Here's another example of a kind of pattern that could be affected:

https://github.com/cubiq/iscroll/blob/v4/src/iscroll-lite.js#L207

that.startTime = e.timeStamp || Date.now();

This is from v4 of Cubiq iscroll, which was both a pretty popular solution for a little while, and used in a lot of websites that are no longer maintained.

I'm not sure if this is actually a problem (I can't tell if the associated method is ever invoked synthetically, perhaps for initialization), but neither can simple static analysis.

Related: "it's often broken in Firefox" doesn't affect mobile websites much, which heavily use e.timeStamp together with touch events for various time-sensitive operations. All that'll happen is that some mobile website that hasn't been updated since 2010 will stop scrolling (or the scrolling will get janky, or the momentum will be horribly off), and who's going to report that, and to whom?

from dom.

I think I'm convinced we should do this via a new property instead given the reported subtle breakage.

However, #23 (comment) mentions that timeStamp is also not quite implemented in the same way across browsers so I guess whatever we do that'll need to change a little bit.

@majido, @birtles, what do you think?

from dom.

Hi,

I think @RByers summarized our thinking and the background pretty well in #23 (comment)

It is difficult (impossible?) to actually measure the impact before hand in this case and our hope has been that once we hit beta we receive better feedback which we are getting.

However, I also did an analysis using very recent (Jan 15, 2016 dataset) httparchive data which shows the impact is likely to be < 0.02%. I looked for the following patterns in top 477k top sites (both mobile and desktop) and tried to categorize and understand if they get broken and I have assumed breakage if the usage was not obvious.

• e.timeStamp - value
• value - e.timeStamp

Clearly this is not the complete picture but gives a sense that the usage of this pattern is quite limited.

In cases where I have seen the issue in a library and have outreached (eg, 1 and 2) the fix has been trivial and has address an error or a workaround in the logic for Firefox.

from dom.

Given how timeStamp cannot really be used compatibly across browsers anyway, and the fact that this has hit Chrome 49 without needing to be reverted, I think we should stay the course and fix timeStamp instead of inventing a new property and also incurring all the risk involved in fixing timeStamp to be cross-browser compatible.

from dom.

At the moment I tend to agree with @domenic. Based on current data and our beta experience until now the impact appears to be limited in scope (although subtle) and within our expected range. Not sure if we should change our strategy yet.

To be fair, most of the code that tries to compare Event.timeStamp with Date.now() is already broken in subtle and not so subtle ways:

• Not dealing with system wall time skews
• Improperly handling firefox edge cases (e.g., In Firefox value may be: zero, a unix epoch but in microseconds, or millisecond value in system startup time).
• Invalid comparison between monotonic and non-monotonic clocks In Safari which can lead to subtle bugs. (In safari, the event timestamp is monotonic but with unix epoch timebase )

from dom.

I want to reiterate a few things (excuse me for being brief as I am on mobile):

shows the impact is likely to be < 0.02%

That appears to be higher than @RByers's proposed threshold of 0.01% (which is 1 in 10,000 sites):

If we find a pattern that, say, we find used on >0.01% of the top sites in httparchive then we should definitely reconsider.

Also, the analysis:

• e.timeStamp - value
• value - e.timeStamp

That does not cover any of the reported cases, including Angular's use case.

...and I have assumed breakage if the usage was not obvious.

None of the reported cases hard errors and only break in ways that cannot be easily detected.

...the fix has been trivial...

Unfortunately, fixing a library going forward doesn't affect existing deployments. It by definition is irrelevant to web compat.

So, I don't agree that we have data to prove that this doesn't matter, even if it has shipped in Chrome beta.

from dom.

It's worth reiterating that the breakage in Angular (which was in code in the main angular.js), would not have been detected by the already-done static analysis.

It also would not have detected the bug that @chancancode and I found in our demo code (we stored e.timestamp in an object).

It also seems that this change breaks at least some part of YouTube, which is still not fixed as of 3 days ago, per comment 24 on chrome bug 574514:

Yes, I am the right owner to fix the test. This test is to ensure that browsers don't break YouTube. I'm going to wait until YouTube fixes their code before I fix the test. As of right now, the event.timeStamp epoch is still a requirement for YouTube.

Even thought it didn't detect these cases, the number of cases detected by simplistic static analysis still exceeds the speculative threshold of 0.01.

While 0.01% sounds like a small number, it means that it affects 1 / 10,000 sites. The static analysis also breaks out "bugsnag". Bugsnag is an error reporting tool that the static analysis suggests accounts for another 0.01% on its own. Again, this is just a case that happened to be detected using the static analysis.

Timing-related bugs are subtle and hard to track down, and this particular change will affect old sites which aren't even maintained anymore. Given that a number of high-profile breakages have already been reported, and that it doesn't manifest as a hard error, I'm having a hard time understanding why the experiment has not conclusively failed.

from dom.

Part of the reason is because those usages already failed in Firefox (in many cases) and Safari (in fewer cases, but more subtly). They're clearly survivable.

from dom.

Given how timeStamp cannot really be used compatibly across browsers anyway

I also wanted to add that this is simply not correct. While timeStamp cannot be used compatibly across browsers in a completely generic way (say, a library operating on all events), there is a large subset of cases that has historically been compatible, and which web content relies on.

The fact that some events do not fit in this subset does not mean we can break the events that do.

I would like someone to tell me what rules I should apply in the future when dealing with web compat issues, because frankly, this thread has made me question whether I understand the rulebook we're supposed to be following.

from dom.

Part of the reason is because those usages already failed in Firefox (in many cases) and Safari (in fewer cases, but more subtly). They're clearly survivable.

"clearly survivable" is overstating the case. The fact that some cases of e.timeStamp didn't work in Firefox didn't stop Angular animations or YouTube from working reliably on Firefox.

from dom.

the fact that this has hit Chrome 49 without needing to be reverted

This seems somewhat self-prepatuating, especially since issues have been (and continue to be raised). So by ignoring issues, yes chrome has not needed to revert...

Part of the reason is because those usages already failed in Firefox (in many cases) and Safari (in fewer cases, but more subtly). They're clearly survivable.

A path of success was possible, but that changed with Chrome 49.

It seems transitioning to high-precision timer since posix epoch may address the concerns being raised, with what seems like little cost? Maybe I'm missing something...

from dom.

It seems transitioning to high-precision timer since posix epoch may address the concerns being raised, with what seems like little cost? Maybe I'm missing something...

In that case the timeStamp value will look similar to Date.now() but only on surface. The actual value is coming from a different monotonic clock which unlike Date.now() is not impacted by NTP time skew. This means that the developers are more likely to compare it with Date.now() which is going to break in subtle ways in random machine which makes the bug hard to detect. To quote dbaron from here

Shipping a feature where a developer getting it wrong means their page is intermittently broken on a small percentage of machines seems pretty bad. Seems like people are likely to get that wrong forever. It seems better to actually go through a more-breaking change once.

If we decide not to change Event.timeStamp then a better solution is to actually use a different attribute.

from dom.

Also, the analysis:

e.timeStamp - value
value - e.timeStamp
That does not cover any of the reported cases, including Angular's use case.
No it does not.

...and I have assumed breakage if the usage was not obvious.
None of the reported cases hard errors and only break in ways that cannot be easily detected.

Sorry, if I was not clear. In this case I meant that if I was not fairly sure about "o.timeStamp - value" usage I assumed it gets broken (i.e., o.timeStamp is an event value and values is a date). To be honest based on the codes I check during the process most of the time this pattern end up being safe, e.g., either timeStamp is coming from Date.now() or value is another timeStamp but I erred on the side of over estimating the breakage in this case.

While 0.01% sounds like a small number, it means that it affects 1 / 10,000 sites. The static analysis also breaks out "bugsnag". Bugsnag is an error reporting tool that the static analysis suggests accounts for another 0.01% on its own. Again, this is just a case that happened to be detected using the static analysis.

In fact, I think 0.01% of the issue being attributed to bugsnag is a promising sign (not to discount other issues). Their usage is essentially what I would categorize under "clearly survivable" category. They usage pattern is to compute a millisecondsAgo for the last event which is reported as part of the metadata for when an error is reported. There is no user-facing impact, and their code was reporting invalid values for Firefox that apparently no one noticed. This is simple to fix and also has also a backend solution. This is the best possible breakage and they seem to be happy about the change.

It's worth reiterating that the breakage in Angular (which was in code in the main angular.js), would have been detected by the already-done static analysis.

I think you meant would NOT have been detected.

It also would not have detected the bug that @chancancode and I found in our demo code (we stored e.timestamp in an object).

Correct.

So the simplistic analysis over counts in some cases and under counts in others. Which is why I don't like to rely too much on it. Just to get a sense of things in combination with other evidence.

from dom.

I think you meant would NOT have been detected.

Right. Fixed :)

from dom.

This is the best possible breakage and they seem to be happy about the change.

What they said was:

That also explains why this feature didn't work reliable on Firefox :)

In other words, they were already aware that their timestamp feature didn't work reliably on Firefox, but didn't know why. The reason they knew that the problem existed at all is because they have a highly generic library that works across events and reports timing, and even they weren't able to track down the details that led to the occasional inconsistency in Firefox.

from dom.

The change has been on Chrome Canary since Dec 3rd, and in Beta channel for more than two weeks now. Here are the reported issues that I am aware of so far:

1. BugSnag, a fairly popular error reporting tool
   Issue: Incorrect "millisecondsAgo" is reported to the backend server for last event that occurred before error.
   Firefox workaround: None, already broken on Firefox.
   Status: Developer is looking into a fix. The data may be cleaned up in the backend.
2. Angular CSS animation module
   Issue: If Angular CSS animation is used that has a 'start-delay', it may end later that expected.
   Firefox workaround: They were using timeStamp = e.timeStamp || Date.now() pattern that depends on animation event timeStamp being 0 in Firefox.
   Status: Fixed on trunk. Other simple workarounds exist.
3. Youtube HTML5 player
   Issue: Event timeStamp is used to measure event processing latency. The event is ignored similar to their behaviour on Firefox.
   Firefox workaround: If timeStamp is not within expected range it is ignored.
   Status: Fix in progress. High-res timeStamp will more accurately reflect the actual processing latency.
4. Plus-for-Trello Chrome Extension
   Issue: ????
   Firefox workaround: N/A, extension was chrome only
   Status: Fixed.
5. Glimmer demo code
   Issue: ???

Are there other issues that Firefox folks (@birtles is that you?) are aware of.
Also more importantly I like to know their opinion and insight on the scope and impact of web-compat issues so far.

IMO @domenic is right to categorize these as survivable
(i.e., small in scope with little or no impact on user-facing functionality) and that it is worth paying small cost of breaking web-compat here to avoid having two timestamps attributes with different semantics for foreseeable future specially given all the issues that currently plague event.timeStamp (A few are listed here).

from dom.

BTW, I also looked at the that.startTime = e.timeStamp || Date.now(); usage pattern mentioned earlier.

This pattern is workaround for a Firefox bug where the timestamp for some events (most notably animation/transition events) can be 0. (I believe this issue is getting fixed in Firefox in conjunction with switching the timeStamp to high-res time value but @birtles is the expert). Some notes:

• The workaround is only ever useful in Firefox.
• The workaround allows the animation/transition events timestamp to become comparable to Date.now() [Note 1].
• The workaround does not make timestamp of any other event (notably input events [Note 2])
  comparable with Date.now().

The workaround is useful for animation/transition events but it is otherwise fragile and prune to bug
if indiscriminately used on all events. It will not protect code that blindly compares
event.timeStamp to Date.now() for all event types.

Looking at the linked iscroll-lite.js, the library is using the above workaround on input events which is incorrect 👎 but fortunately they only compare startTime to other event.timestamp values and never to Date.now():+1:. So despite using the workaround unnecessarily (incorrectly?) the code actually works fine and should not be impacted by switching to DOMHighResTimeStamp.

Note 1: This is the Angular css-animation usage pattern which was impacted by the switch.
Note 2: In Firefox, input events do get a non-zero timeStamp with a system startup timebase that is incompatible with Date.now().
So for input events, this workaround is not actually exercised and if the code ever tries to compare an input event timeStamp
with Date.now() it breaks.

from dom.

Firefox workaround: None, already broken on Firefox.

People have said this a few times on the thread but this isn't quite right.

Some Firefox events in some cases didn't provide the information. There is a long-running thread (opened in 2001!) about it, and Firefox slowly but surely fixed many of these cases.

DOM3 allowed Firefox to return 0 for "not supported", which they did, and jQuery fixed it to normalize it to Date.now(). In practice, this means that "Firefox returned 0, therefore it was broken anyway", isn't actually true with any code that uses jQuery.

To recap: Chrome, Safari, and IE reliably (at least recently?) return Date.now() for e.timeStamp, and the inconsistency has been fixed by jQuery, leaking the assumption of "since 1970" to the web.

jQuery documents this assumption and DOM4 canonized it, which Firefox noticed in 2011, which caused them to change their implementations for many events.

I also just want to reiterate that the fact that some cases don't work on Firefox doesn't take away from the fact that some cases do! It's these cases that are causing the compatibility hazard.

from dom.

@RByers you said:

Firefox never implemented the UNIX-epoch version, so felt changing the timebase was already largely web compatible and understandably didn't want to regress to using wall-times in a property that was already (rightly) monotonic for them.

But I'm not aware of Firefox necessarily using a monotonic clock for event timestamps. I think we did in some (very few) cases, sometimes along with having the timestamp be in microseconds instead of milliseconds, mostly used wall-clock UNIX-epoch, and sometimes used 0...

from dom.

Firefox never implemented the UNIX-epoch version, so felt changing the timebase was already largely web compatible and understandably didn't want to regress to using wall-times in a property that was already (rightly) monotonic for them.

But I'm not aware of Firefox necessarily using a monotonic clock for event timestamps. I think we did in some (very few) cases, sometimes along with having the timestamp be in microseconds instead of milliseconds, mostly used wall-clock UNIX-epoch, and sometimes used 0...

Firefox Event.timeStamp is a mixed bag. Here is what I have gathered based on my tests on Firefox which is also close to what is claimed here:
1- zero: animation, transition, load events, scroll, resize, focus, blur, drag/drop, dblclick
2- monotonic (?) since system startup in milliseconds: mouse, wheel, key, touch, click events
3- epoch in milliseconds: composition events, 'input' event, change
4- epoch in microseconds: custom events, hashchange, popstate

But I'm not aware of Firefox necessarily using a monotonic clock for event timestamps. I think we did in some (very few) cases

All mouse, wheel, key, touch, click events seem to use a monotonic clock timestamp. I wouldn't call this very few.

from dom.

Firefox workaround: None, already broken on Firefox.

People have said this a few times on the thread but this isn't quite right.

In this case, my comment was specific to BugSnag which does not use the workaround for zero timestamp making it broken on Firefox for the vast majority of events. I have specifically called out the zero timeStamp workaround where it has been applicable.

I also just want to reiterate that the fact that some cases don't work on Firefox doesn't take away from the fact that some cases do! It's these cases that are causing the compatibility hazard.

I agree. Note that these only become a compatibility hazard if compared with another epoch timeStamp (i.e., not coming from another event). This appears to be a minority usecase for Event.timeStamp based on what we have seen so far.

from dom.

2- monotonic (?) since system startup in milliseconds: mouse, wheel, key, touch, click events

Hmm. Looking more carefully, I agree that mouse events (the ones I checked) are monotonic on at least some OSes for Firefox. And it even looks like they're now all in ms (this did not use to be the case for the function they're using).

But note that this is not the same monotonic clock as the one used for performance.now() and hence can skew from that clock anyway...

from dom.

High-resolution Event.prototype.timeStamp is now available by default in Chrome 49 and Opera 36 (stable).

from dom.

This has successfully shipped in Chrome and Opera with very little compat fallout - it seems clear it has stuck. Any concern with updating the spec to reflect that timestamp is a DOMHighResTimestamp and reflects the time of the earliest known source of the event?

from dom.

We don't know yet what the compat fallout is yet.

Changing the spec from normative milliseconds since epoch (the status quo) to a different timestamp should probably wait for more browsers to ship the breaking change and more analysis of the consequences.

If things go wrong, this particular change is unlikely to trigger a wave of complaints, but rather a trickle of confused users and subtly broken sites. Let's see what happens.

from dom.

If things go wrong, this particular change is unlikely to trigger a wave of complaints, but rather a trickle of confused users and subtly broken sites. L

^^

from dom.

Happy to wait for another browser with an independent engine to ship this, perhaps coupled with a few months delay depending on how fast that will be.

from dom.

@DigiTec @rniwa @birtles, any thoughts on making the same change to your respective browser engines?

@annevk, maybe add a note in the spec that this is a change-in-progress to help implementors discover this?

from dom.

Yes, we're planning to make this change. I'm just waiting on @kentuckyfriedtakahe to help with the platform work.

from dom.

@birtles, it is not very high on my priority list because YouTube (who originally asked me for it) implemented a work around

from dom.

Note that it's probably necessary for effectively monitoring input
performance, like http://rbyers.net/scroll-latency.html which we will be
encouraging developers to do.
On Apr 21, 2016 3:26 PM, "Anthony Jones" [email protected] wrote:

@birtles https://github.com/birtles, it is not very high on my priority
list because YouTube (who originally asked me for it) implemented a work
around

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#23 (comment)

from dom.

I don't think we (WebKit) would be comfortable making this change given the number of compatibility issues being reported here.

from dom.

@birtles I would really appreciate an opportunity to advocate against this change somewhere where the full rationale of the Mozilla position is presented. I think it's a mistake and I don't agree that Chrome has empirically come close to showing that it's harmless, even after shipping.

from dom.

@wycats: @bzbarsky, @annevk and I discussed this in February this year and I think @bzbarsky in particular had a preference for adding another eventTime or time member given the compatibility concerns.

However, since then Chrome has shipped with reportedly few issues so I was under the impression that this was proving to be Web-compatible and the extra member was not required. We can revisit that if there indications to the contrary, however. Although perhaps your point is that the burden of proof lies with those shipping?

(For the Mozilla position, that dates back to bug 77992 which lead to me proposing to add Event.creationTime to whatwg, from where the suggestion came that we just try to change timeStamp and see if it works.)

from dom.

However, since then Chrome has shipped with reportedly few issues so I was under the impression that this was proving to be Web-compatible and the extra member was not required.

The very fact YouTube had to workaround this and Angular had to have a fix is a good indication that this change is not Web compatible. Just the issues listed in #23 (comment) are enough indication to us that we can't make this change in the foreseeable future.

from dom.

@birtles I have a reduced test case that I think illustrates the kinds of patterns that wouldn't be web-compatible, despite the apparent weirdness in Firefox:

<html>
<body>
  <div id="output"></div>
  <input id="textbox" type="text" placeholder="focus here">
  <script>
    document.getElementById("textbox").onfocus = function(e) {
      var date = new Date();
      date.setTime(e.timeStamp || new Date());
      document.getElementById("output").innerHTML = "<p>" + date + "</p>";
    };
  </script>
</body>
</html>

This code worked in all browsers until the latest Chrome change. It relies on the fact that, for many events, the timeStamp is (interoperably) either the same as new Date() (as required by DOM3 and DOM4) or 0 (as allowed by DOM2).

This reproduction is terse and works on all browsers other than the latest Chrome for all events that reliably provide e.timeStamp as either the DOM4 specified value or 0 (and there are many such events).

Some things to note:

Firefox's "Zero"

While Firefox has, for a long time, returned 0 for many events, web developers learned to say || new Date() to coerce the zero into a "good enough" date value.

This pattern is quite common (scan through this GitHub search including in highly popular libraries.

The Angular bug was exactly this situation. Here's the fix:

- var timeStamp = ev.$manualTimeStamp || ev.timeStamp || Date.now();
+ var timeStamp = ev.$manualTimeStamp || Date.now();

setTime

It's easy to observe that developers expect the result of e.timeStamp to return a value that looks like a JavaScript date, because date.setTime(e.timeStamp) is very common.

As people have pointed out, that didn't always work on Firefox, but the || new Date() workaround is shockingly common once people discover the problem. In fact, it's been a part of jQuery forever as part of its "event fix-up logic":

this.timeStamp = src && src.timeStamp || jQuery.now();

Here it is in the latest version of jQuery and here it is in 2011.

In short, the existence of the || now() and its adoption by jQuery means that the events that return 0 in Firefox still have a terse, common, and interoperable semantics that web developers depend on.

Comparing to Fixed Times

Finally, Chrome seems to believe that the only valid comparison of an e.timeStamp is another e.timeStamp. In fact, there are two other valid comparisons:

• comparing e.timeStamp to a fixed time in the past (animation start, app load, etc.)
• comparing a previous e.timeStamp to the current time, when not directly inside an event handler.

These problems figured into virtually all of the already-known reproductions, and it's simply reasonable for developers to want to compare against a fixed time. Since performance.now() is relatively new, any developer in the past who wanted to compare a timestamp to a fixed timestamp had no choice but to compare to Date.now(), and that code is both reasonable and often not very easy to fix.

I would encourage people who believe that shipping in Chrome proves that the problem doesn't exist to search Stack Overflow and other avenues for people confused about the breakage in Chrome. It will only increase over time, and people asking questions only represent a small percentage of total web content with the breakage.

The Standards Status Quo

DOM2 also allowed for a flexible epoch, but in practice a large percentage of all events were supplied (again, interoperably) as either milliseconds since 0:0:0 UTC 1st January 1970 or 0. Before the recent change in Chrome, Firefox developers had noticed that DOM4 now required a unix-epoch-based timestamp, and jQuery filed a bug against Firefox asking to fix the remaining events, which was closed as a duplicate of the bug tracking DOM4 behavior.

The TLDR is that until Chrome decided to make this change, the DOM spec had been increasing in precision, and browsers were coming into compliance.

DOM2

Used to specify the time (in milliseconds relative to the epoch) at which the event was created. Due to the fact that some systems may not provide this information the value of timeStamp may be not available for all events. When not available, a value of 0 will be returned. Examples of epoch time are the time of the system start or 0:0:0 UTC 1st January 1970.

DOM3

Used to specify the time at which the event was created in milliseconds relative to 1970-01-01T00:00:00Z.

DOM4

The timeStamp attribute must return the value it was initialized to. When an event is created the attribute must be initialized to the number of milliseconds that have passed since 00:00:00 UTC on 1 January 1970, ignoring leap seconds.

from dom.

@birtles:

@wycats: @bzbarsky, @annevk and I discussed this in February this year and I think @bzbarsky in particular had a preference for adding another eventTime or time member given the compatibility concerns.

I wish this was shared earlier. All other signals so far has been that Firefox is on-board with
the plan to experiment and ship if we don't find a non-trivial compat issues. In fact originally
we were happy to wait for Firefox to ship and report on compat issue before we go ahead.

@rniwa:

However, since then Chrome has shipped with reportedly few issues so I was under the impression that this was proving to be Web-compatible and the extra member was not required.

Well the list of reported issues is essentially the same as before even after being in Chrome stable (on all platforms) for the past 6 weeks. IMHO this is very good evidence. Moreover the result matches our expectation based on the httparchive data analysis we did before launch which is another good signal.

The very fact YouTube had to workaround this and Angular had to have a fix is a good indication that this change is not Web compatible. Just the issues listed in #23 (comment) are enough indication to us that we can't make this change in the foreseeable future.

We have expected some web-compat issues all along, the question has always been the scale and the impact of such issues. We have attempted to get a sense of this by using httparchive and carefully monitoring beta, dev channels before launch and have continued this after going to stable.

On YouTube case: They had an existing workaround for Firefox edgecases which works
well for Chrome recent changes as well. So they haven't done any modification in response to a breakage.
So there was no modification or impact.

On Angular CSS Animation module: The scale and impact is quite low here. See detailed
description of the issue. Happy to debate my evaluation of impact though.

@wycats:

Thanks for the writing this up. It is a good summary of differences between browsers and main
workarounds being used. I have some notes though:

for many events, the timeStamp is (interoperably) either the same as new Date() (as required by DOM3 and DOM4) or 0 (as allowed by DOM2).

I have an fairly accurate list of the Firefox behaviour. It is a mixed bag. For many events this is not true (including almost all input events):
mouse, wheel, key, touch, click events, hashchange, popstate, any custom events

It's easy to observe that developers expect the result of e.timeStamp to return a value that
looks like a JavaScript date, because date.setTime(e.timeStamp) is very common.

It is actually very hard to get any reasonable signal from the above github search.
I looked at the first 10 pages. Lots of false positive (timestamp not related to any event) and
mostly dominated by a single code snippet from a tutorial set. Not really indicative of any real usage.

any developer in the past who wanted to compare a timestamp to a fixed timestamp had no choice but to compare to Date.now(), and that code is both reasonable and often not very easy to fix.

This is rather trivial to fix actually. (Use Date.now() in event handler than timestamp)
Also how big of a usecase is this? Our httparchive search shows it is rather small.
Finally this pattern has subtle bugs and not very dependable even after Firefox workaround.

The TLDR is that until Chrome decided to make this change, the DOM spec had been increasing in precision, and browsers were coming into compliance.

Coming into compliance since 2011 (DOM3) and we still don't have that. This is what worries me
about introducing yet another timestamp attribute with a slightly different semantic. Now we will have two slightly in compliance values to take care an push forward in the platform not to mention the burden that it has on API complexity and long-term simplicity of the platform.

It is this option that we have to weigh against web-compat issues. Compat issues that our data so
far suggests them to be small in scale, easy to fix, low impact, and subtly broken already. We still have a good chance to make the switch without incurring the cost of two timestamp attributes.

Finally, from Blink side we are happy to follow up with any new consensus on this issue. We have
been active in pursuing this and have tried to help reach a reasonable inter-operable solution in
good faith :).

from dom.

I talked with others at Apple and we're pretty sure we'd never make this change as it's not web compatible. The only acceptable change would be adding a new propety for us.

from dom.

@birtles:

@wycats: @bzbarsky, @annevk and I discussed this in February this year and I think @bzbarsky in particular had a preference for adding another eventTime or time member given the compatibility concerns.

@majido:

I wish this was shared earlier. All other signals so far has been that Firefox is on-board with the plan to experiment and ship if we don't find a non-trivial compat issues.

TLDR: By February, multiple Mozilla engineers had become increasingly uncomfortable with the change and repeatedly described it as not-web-compatible.

The concerns about web compatibility were repeatedly shared earlier, around the same Feburary timeframe. Based on the threads you and @RByers participated in, I'm not sure where you are getting the idea that Firefox was still on board with the plan.

On December 10, in a linked issue from this thread:

We have a web compat issue reported in bug 1231619. See bug 1231619 comment 3.

In the referenced bug bug 1231619:

So this is an actual web compat problem.

At this point, Mozilla engineers were starting to think that there might be a web compat issue.

@RByers replied:

Personally I don't think this example alone should block our plans to ship DOM highRes event timestamps.

This is the first of many comments by Chrome engineers waving away reports of web compatibilities.

@birtles:

Clearing needinfo since it's clearly a Web compat problem.

@dbaron:

Web compatibility issue with a well-known library that we need to avoid shipping.

@RByers then tried to persuade Firefox to ship anyway. @dbaron replied:

Shipping a feature where a developer getting it wrong means their page is intermittently broken on a small percentage of machines seems pretty bad. Seems like people are likely to get that wrong forever. It seems better to actually go through a more-breaking change once.

Again, Mozilla engineers are becoming increasingly concerned, contrary to the idea that they were still just fine with the plan.

You replied, arguing that since Angular fixed the issue, we're clear to ship:

FYI, Angular has fixed the issue[1] and it is expected to be in their upcoming 1.5 release.

Around this time, @chancancode and I noticed that this problem broke one of our demos. @stefanpenner (another Ember developer) reported the "bug" to Chrome, and @chancancode elaborated:

Again, once we identified the problem, it was an easy fix, but I'm just quite worried about web-compat given the subtle nature of this difference.

@majido replied:

This is indeed the new desired behaviour which is currently planned to be launched on M49. See: https://www.chromestatus.com/features/5523910145605632

Again, no major concern that there might be a web compatibility issue, but rather "damn the torpedoes; full speed ahead!" and a suggestion that we should work harder to fix the problem on our end. We were happy to do so, but concerned about the web compat issue.

Since we quickly discovered the Angular thread and noticed that @majido and @RByers were on it, we moved the conversation there.

I believe there is a more general form of this problem that is likely to break other sites.

You replied:

I prefer that we try to use monotonic time first.

@chancancode replied with a very detailed comment explaining our concerns.

At this point, it appears that bz became increasingly concerned:

Fwiw, I generally agree with comment 19.

Around the same time that @chancancode and I reported the bug, YouTube engineers discovered a bug in their test suite.

The code in question was relatively simple:

var last = Date.now();
// later
video.addEventListener('timeupdate', function(e) {
  runner.checkGE(e.timeStamp, last, 'event.timeStamp');
  last = e.timeStamp;
});

In other words a classic example of wanting to compare an e.timeStamp against some earlier fixed time.

The YouTube engineer originally surmised:

This is a browser bug.

At this point, with the breakage in Angular, the report we had of a bug in a demo, and this bug by YouTube, it should have become clear that this change was not web-compatible. Instead of easing off and trying to learn more, the Chrome team pressed ahead, sure that they could just wish the problem away.

@majido told the YouTube engineer:

Sure, but there is a planned changed for DOM spec: #23
And Chrome M50 will reflect this change. As #16 pointed out the spec update is pending us successful shipment in Chrome.

The engineer replied:

Yes, I am the right owner to fix the test. This test is to ensure that browsers don't break YouTube. I'm going to wait until YouTube fixes their code before I fix the test. As of right now, the event.timeStamp epoch is still a requirement for YouTube.

Again, the fact that YouTube made this "mistake" should have been enough evidence to ease off and get more information.

Now that Chrome has shipped, you're saying things like:

On YouTube case: They had an existing workaround for Firefox edgecases which works
well for Chrome recent changes as well. So they haven't done any modification in response to a breakage. So there was no modification or impact.

I can't see the YouTube source code, but after Chrome shipped, YouTube did update the test in question to stop relying on e.timeStamp being directly comparable to Date.now() with the message "Updated EventTimestamp test on Tip to be more spec compliant and Chrome compliant".

I'm not sure where you got the idea that everyone was just fine with moving ahead, but it does seem clear that you deeply wished for that to be true.

It continues to be the case that textual analysis is very unlikely to identify most sources of this bug, and that incidents of the bug are likely to be under-reported (people will believe that the web is janky, rather than that there is a bug in the content they're looking at).

from dom.

One point of record: @dbaron's quote above wrt to a more-breaking change was a response to a suggestion that we define event.timeStamp in terms of performance.now() + navigationStart, more or less, so it looks comparable to Date.now() values. But isn't really, as clocks drift during the lifetime of the page. He was saying he would prefer just having event.timeStamp not be comparable to Date.now() at all to that setup.

from dom.

Ok let's take the rhetoric down a notch, we're all trying to do the best thing for the web platform.

There was general agreement that (in order to resolve the multi-year debate about whether a new API is necessary or not) Chrome should attempt to ship this change. Once we started down that path and were getting concrete data, our judgement (vocal opposition notwithstanding) was that the compat issues we were seeing were minor and the short-term cost caused was well below the long-term cost to the platform of introducing a confusing and largely redundant API. So we did not see a need to reverse course. But I recognize this is entirely a judgement call where opinions of reasonable engineers will differ.

If Mozilla and Apple now feel that, despite not having concrete examples of real sites that are broken today in Chrome, there still needs to be a new API for this, then by all means please spec such an API, ship it in at least one of your engines, and we'll be happy to follow suit. We really do want to see the problem of reliably measuring input event latency solved once and for all in an interoperable way.

from dom.

It seems that from #23 (comment) the current requirement in DOM for timeStamp is not onerous and can be implemented across browsers.

It's not clear if we want to store two different times for each event, forever (different in both origin and resolution). (Though Apple seems okay with this given @rniwa's comment.)

It's also not clear what the compatibility impact is of changing timeStamp, exactly, though with Chrome shipping the change it cannot be huge, I think. Though as suggested earlier we might know more in a couple of months.

It seems unlikely other vendors will ship something fast, given #23 (comment) and #23 (comment), so maybe the best approach here is to just wait it out a bit and see what actually happens.

from dom.

It seems unlikely other vendors will ship something fast, given #23 (comment) and #23 (comment), maybe the best approach here is to just wait it out a bit and see what actually happens.

Actually @rniwa didn't say anything about the priority they'd give to landing a new API in WebKit here. @rniwa thoughts?

Note that with passive event listeners now shipping, we're starting to actively encourage the measurement of scroll latency using this API. But all our examples will show a pattern that works for both the WebKit and blink implementations (the only two implementations today exposing the original driver time), so it'll probably still be possible for us to switch back to the WebKit behavior in the future without causing too much disruption (it'll just introduce the NTP skew problem in scenarios that were previously more stable in Chrome, but we don't know how bad that tends to be in practice).

So if collecting more data on the impact in Chrome over a few months would be valuable to this debate, then that's fine with me. But if no amount of such additional data would change other implementor's minds then there's no benefit to waiting.

from dom.

Ok let's take the rhetoric down a notch, we're all trying to do the best thing for the web platform.

I apologize for the rhetoric, and I agree fully that everyone here is trying to do what they believe is best for the platform and the browsers they work on.

So if collecting more data on the impact in Chrome over a few months would be valuable to this debate, then that's fine with me.

I think more data would indeed be useful, if we measure the right things.

Here are some things that (dynamic) telemetry might be helpful if unearthing, if you're willing to run it:

• for each event:
  • comparisons of the value produced by e.timeStamp with the value produced by Date.now().
  • whether the timestamp used in such those comparisons came from e.timeStamp || Date.now(). For events where Firefox produced ms-since-unix-epoch or 0, this would have previously been guaranteed to produce a date, but now it does not.
  • runtime behavior equivalent to e.timeStamp || Date.now() (there are many ways to do this coercion, so a textual analysis is not enough).
  • usage of date.setTime(timestamp) or date.setTime(coercedTimestamp)

I'd be happy to discuss the exact telemetry that would be most helpful here, if you find this line of reasoning useful.

I'd also add that finding "hits" with this kind of telemetry does not prove that a site is broken, but it will give us a corpus of sites to evaluate manually.

Personally, I'm especially concerned about the vast number of "iOS lookalike" sites that were hand-crafted early in the days of iOS, because those sites were forced to reimplement scrolling, implement high-level touch events, implement animations and more, without the benefit of window.performance.now() even existing at all to give them a clue that they should avoid comparing e.timeStamp with Date.now().

Also, because those sites are largely "webkit only" in practice, they may well not have noticed the Firefox breakage at all, but these sites are on the Internet and most browsers have been attempting to support them.

from dom.

Just some data, event.timeStamp has been a PITA to jQuery for a long time since people expected us to fix it. It was broken in Firefox until 2014 and broken in Opera until 2012. We also received a report in 2012 that it was broken on Android 2.3 Dolphin. So anyone depending on reliable cross-browser results has been getting wrong answers until the last 3 or 4 years.

jQuery used to try patching this problem by filling in a Date.now() value for the timestamp if it was zero but removed that code in jQuery 1.7 (January 2011). I hope that encouraged people to stop using the value altogether, but you can still find various GitHub code and StackOverflow answers that assume the timestamp is compatible with Date.now().

from dom.

I had a good chat with @majido off-thread and we came up with some good telemetry that we agree would identify the impact of this change. I'm looking forward to the results!

from dom.

It was a good chat indeed. :)

I am trying to see if I can gather more data using telemetry to better help make progress on the debate about the potential impact. This should help address some cases where our earlier static analysis of httparchive would have missed. I can only do this mainly thanks to @RByers recent efforts to enable cluster telemetry for Blink that allows such measurement for a set of top sites (e.g, top 10K).

Following @wycats suggestions I think we should measure the following cases:

1. anyDate - timestamp or timestamp - anyDate
2. new Date(timestamp)
3. date.setTime(timestamp)

Where timestamp is a value that came from Event.timeStamp **

As pointed out earlier, this will over count because there is no guarantee that any of the operations above will actually lead to breakage but we can followup with a manual check on this smaller subset. I am discussing with our V8 experts to see if this is actually possible without a huge amount of work. I will update once I have an answer.

In any case, before spending too much effort on this I like to have some agreement that this is an acceptable experiments and an acceptable threshold (e.g., 0.02% of pages?) for seeing this type of usage leading to breakage.

Also particularly interested on what @rniwa think of the effectiveness of this measurements given his strong concerns on web-compat.

** I have not yet figured exactly how to tag timestamp and Date values as I believe they get converted into a numbers before being operated on. Perhaps I will use a special value (may impact program flow) or specific fractions (A lot less intrusive)

from dom.

In any case, before spending too much effort on this I like to have some agreement that this is an acceptable experiments and an acceptable threshold (e.g., 0.02% of pages?) for seeing this type of usage leading to breakage.

Generally speaking, I have concerns about treating any specific percentage of pages as evidence that breakage is ok. Because the web is so huge, "real" applications are a very small percentage. Maybe we could categorize breakage by things like presence of certain globals (Ember, Backbone, React), usage of other "tracer" features like history.pushState that would indicate that a website is more of an "application" than pure content?

If you're interested, I would love to help beef up Blink's usage analysis in this way ("application" is one useful category, but there are probably others, like "mobile site", etc.). It would make me feel a lot more confident that the identified features aren't heavily used in some pocket of the web.

from dom.

But I would also like to add that I think we can come up with a measurement that I could agree ahead of time would be enough (just not "total usage over the entire web").

from dom.

But I would also like to add that I think we can come up with a measurement that I could agree ahead of time would be enough (just not "total usage over the entire web").

If I use cluster telemetry, the corpus will be pretty well defined (e.g. top 10K sites) rather than the entire web and I can better control for mobile vs desktop. Though it also means I can only test a pre-defined set of interactions but has the benefit of being able to get the URL of the page.

If I end up being able to get some type of safe measurement in Canary then it will be website used by Chrome Canary users. This is more like the entire web but page URL will not be available.

from dom.

If you're interested, I would love to help beef up Blink's usage analysis in this way ("application" is one useful category, but there are probably others, like "mobile site", etc.). It would make me feel a lot more confident that the identified features aren't heavily used in some pocket of the web.

Yeah this is a problem the blink API owners (especially @foolip and I) have spent some time worrying about, but don't yet have any super-promising practical ideas around. We should think about it further and we'd love your help! We're ramping up our investments in our compat tooling, deprecation process, interop efforts, etc. I filed this chromium bug to track.

from dom.

@dmethvin Thanks for the jQuery background. Current Event.timeStamp is PITA indeed. Hopefully we get it fixed one way or another soon.

Even today despite my experience using Event.timeStamp I got surprised seeing negative values (up to -300ms) for Date.now() - event.timeStamp for input events in Safari on Mac OS (demo page) /cc @rniwa. It is a small issue (most probably due to an NTP skew) compared to others but can lead to subtle head scratching bugs.

from dom.

There was some discussion of this at TPAC:

• @smaug---- said he wants to see this change land in Gecko as well (really would rather not have a new property, leaving the old confusing one indefinitely)
• @n8schloss said having these high-res timestamps in Chrome has been very valuable to Facebook in their scroll performance investigations
• A couple Edge perf folks said they would look into changing Edge for this (but aren't able to comment on this issue since it's in the WHATWG)
• @rniwa indicated that if Gecko and/or Edge shipped this change alright, then perhaps WebKit may be able to as well but that @smfr was the one to talk to.

from dom.

FYI, bug 1026804 got fixed two days ago.

Sebastian

from dom.

Okay, so with Chrome and Firefox shipping, these things still need to be done:

• Bugs filed against Edge and WebKit.
• Make sure there are sufficient web-platform-tests.
• Prepare a pull request for the DOM Standard.

I'm happy to help folks if they want to take on one or more of these tasks.

from dom.

Edge: https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/12726672/

from dom.