Question - differences between web-vitals library, web-vitals extension, and CrUX report about web-vitals HOT 13 CLOSED

But, don't know if you are aware, but CrUX measure performance based on pathname, which means it mix results from LoggedIn and LoggedOut users. Also the CrUX dataset will always be a subset of your production field data, as not every user in Chrome gives permission to collect and report those metrics back to CrUX.

More to read, here.

So whatever you will measure in the field based on your production user will always diverge from CrUX. cc @rviscomi

Just make sure to use your Field Data as proxy to understand improvements and regression trends in your App. And have CrUX as ultimate validator of your work.

from web-vitals.

Thanks for the cc, @Zizzamia. It's true that CrUX is a sample of the larger visitor population; it only includes Chrome users who opt-in to sharing usage statistics. Still, we've found the sampling is representative and there's a strong correlation between CrUX and other field data sources, so I wouldn't expect to see metrics vary too widely. The web vitals JS library powers the extension and it's built very similarly to how CrUX is measured by Chrome, so especially within those tools the data should be closely correlated.

@skaraman @psabharwal123 could you describe your site architecture and how you're measuring your own field data? It sounds like you're connecting the web vitals JS library to your GA account. I'd also be curious to hear if your site is an SPA, as that can complicate things.

from web-vitals.

I believe this is now covered by this doc: https://web.dev/crux-and-rum-differences/

from web-vitals.

I'm not sure to what you are referring.. I mean, are you referring about the values coming from different sources? they are not similar?

from web-vitals.

@ubertu yes values from different sources are not similar. We are trying to replicate how chrome measures these metrics (since that will be important for SEO) by using this library.

from web-vitals.

web-vitals library will return you a real and full value of the metrics, based on all the real user data. They are evaluated based on all the users the browser is able to calculate them. The others are somehow statistical measurement. Some time based on lab data and sometime based on sampling data. I would say the more accurate your measurement are, the better is, so definitivelly to me what you need is to use the library

from web-vitals.

thanks for the reply, am i correct to understand that for SEO google will use the CrUX data which is also measured in lab?

from web-vitals.

Who knows what google will do.. for sure they have the way to retrieve real data. Told that, using real data will make you in an ideal shape, misregard what they will.use

from web-vitals.

It's not really a question of real data, both this library and CrUX provide real data, this is more of a question of how chrome measures the data, for a Google engineer.

from web-vitals.

The problem is our data shows lcp to be a lot less than from crux, most likely because we have a carousel, so I wanted to understand at what point does chrome send measurements. I understand data from field and crux would not be similar but I would expect some correlation.

On a side note we also measure mix of logged in and logged out users and ga data is also sampled.

from web-vitals.

Sure, our site is a SPA and PWA. FID seems to have a good correlation but LCP and CLS do not. We measure vitals only for the first page load, we send cumulative measurement for each metric once user interacts with the page (scroll/click/touch/etc)

from web-vitals.

How much are the two data sources diverging, quantitatively? I'd be especially interested in the percent of "good" experiences and 75th percentiles for each CWV. And are you able to tell if this applies to all users or only certain segments, like mobile?

Being an SPA means that CrUX is only able to measure first page load experiences, but given that you're comparing it with first page load experiences from your own field data, it shouldn't be affecting the results.

from web-vitals.

sorry for the late reply, they seem to diverge from about 5 to 20%. Our tracking is for all users, mobile desktop, guest, club.
i noticed #75 and #79 which might be affecing us, and #61 would be fixed in chrome 88 i believe which might help

from web-vitals.