[BUG] Duplicate customEvent entries about applicationinsights-js HOT 15 CLOSED

I tried 3.0.7 just now and the dup issue seems to be fixed. Thanks much for tracking this down!

from applicationinsights-js.

Further update: I think I have managed to mitigate this issue by setting disableFlushOnBeforeUnload to true and calling flush in my own onbeforeunload handler:

from applicationinsights-js.

There are many possible reasons for duplication of events, this issue only being one of them (unfortunately).

So can you try using the new disableSendBeaconSplit configuration with v3.0.6 (this stops use from attempting to split up and send individual events if then initial sendBeacon fails (because of the size)

However, this will also cause an XHR request to be sent (during page unload) which may get cancelled by the browser and not tell use that the event sending was successful (because either the browser terminated the request or the page simply navigated away before handling the response -- so we never get told). This later case is the same as what it should have been prior to v3.0.3.

You could also try completly disabling sendBeacon usage during page load (so it just goes straight to an XHR call) with setting onunloadDisableBeacon to true

from applicationinsights-js.

@Karlie-777 can you also please re-look at this to see if we have missed something.

from applicationinsights-js.

One last thing: why don't you set the namePrefix to something unique upon instantiation automatically? Might be something to look into, although there might be good reasons not to do that, too...

This is mostly because the AI_sentBuffer and AI_buffer is where Application Insights "trys" to keep any unsent batched events, so that if the page crashes or navigates away when the user comes back to the page it picks up these events and sends them with the new page.

If we set these to a unique prefix for every page load then it would never (or very rarely) find these old unsent / acknowledged events and send them off (leading to event loss). By using the ping API's we are attempting to send all of the events within the same page context that captured the event to reduce these events (because if the user never comes back or they close all browsers (resulting is the Session based storage getting cleared) then these events are lost. And as one of these sequences may be exceptions there may be lost critical events.

So probably what was happening in your case (with all instances using the same empty prefix) is that on initialization all of them where / are picking up these "batched" events and re-sending them (probably because they could not be sent via the sendBeacon -- because of the limit). Not sure why v2 wasn't also doing this as I don't believe we refactored the buffering... It may be that in v2 we used a synchronous XHR request (which chrome doesn't allow anymore) during the page unload so v2 always marked them as complete -- which would have been silently dropping the events.

from applicationinsights-js.

Update: The duplicates seem to appear when the web browser is closed before the interval causes the queued data to be sent. Perhaps an issue with how it flushes with onbeforeunload?

from applicationinsights-js.

I think this might be related to #2195 where the sendBeacon limit (of 64Kb) is being hit and the code is causing the events to be resent.

You could also downgrade to v3.0.3 instead to avoid this specific issue (assuming that is the issue).

from applicationinsights-js.

This one is not solved in 3.0.6 as far as I can tell.

I am seeing the same duplicates behavior for at least customEvents and dependencies. It is happening on close/reload and does seem to be related to "bad flushing logic", almost as if the flushing doesn't mark records as flushed and they get sent again or something along those lines. If I downgrade to version 3.0.2 it doesn't happen, if I use any version from 3.0.3 up to and including 3.0.6 there is something wrong (but perhaps different things, cause those all had minor funny issues). I am not using any (manual/own) flush in my app.

It is not super easy for me to reproduce or track down sorry, but it is easy for me to find the dups in logs:
union dependencies, customEvents summarize count() by timestamp, id, client_OS, itemType where count_ > 1 order by timestamp desc
shows me all the dups. The dups are exact dups, including timestamp. Dups disappear when I revert to 3.0.2. My app is a PWA/single page app in case that matters. Not really a big problem for me since 3.0.2 works fine, but I hope it can help you track down the issue.

from applicationinsights-js.

I am experiencing a similar behavior when sending custom metrics from my app.

The bug can be easily reproduced with this. Just replace the instrumentation key with your own (look for ). The source code was taken from react app insights app and changed. It is supposed to track custom metrics about the duration the page was open (see implementation of EngagementTracker).

Similarly, flushing the messages with appInsights.flush() fixes the problem.

from applicationinsights-js.

for 3.0.6, try setting

extensionConfig:{AppInsightsChannelPlugin:{
     onunloadDisableFetch: true
   }}, 

in the config

from applicationinsights-js.

I would like to add to this that in our case, the issue is still present in version 3.0.7, but with a very specific setup: We have a dashboard solution that runs many different smaller components that each initialise their own Application Insights 'instance'.

This worked perfectly in version 2.x (we've been using 2.6.3 and 2.8.9 specifically over the past years), but the flushing now appears to be not working the same as before and we see many dupes.

We were using 3.0.6 when I saw the same issue as described bij the OP, so we upgraded to 3.0.7. This did fix the on-unload flushing of duplicate messages when only one component used an App Insights instance, but as soon as we add more components, the problem re-arrises and multiplies each time we add more components to the page that have their own instance.

We use this code to initialise the instance:

By the way, it's not just trackEvent that fires these duplicates, it's also trackException.

Using Fiddler to watch network traffic, I see many duplicate track events when I switch browser tabs back and forth:

The green one is the one that fires normally, the red ones pop-up when I leave the browser tab.

Any hints on what we could do to fix this?

from applicationinsights-js.

Additionally, setting disableFlushOnBeforeUnload to true and doing a flush() immediately after the trackEvent or trackException 'solves' the issue, but we miss out on all the performance improving abilities of the library of course...

from applicationinsights-js.

@MarksPoint please try

1. onunloadDisableFetch: true and disableSendBeaconSplit: true.
2. please try setting namePrefix = "yourinstancePrefix" for each instance (the nameprefix for each instance should be different)

from applicationinsights-js.

For any of the "automatic" events (RemoteDependency (Ajax (XHR/fetch) requests) unless disabled or excluded and runtime exceptions window.onerror) by default every SDK instance will receive notification when these occur they WILL also be sent an event (they will contain the same content, but they are different objects (with different instrumentationKeys)) -- so that "might" explain the trackException (unless your calling appInsights.trackException manually -- in which case only that instance will receive the event).

On the flushing situation, try what @Karlie-777 has suggested as those configurations (should) be reverting back to the same operation as v2 by default (don't use sendBeacon() or fetch (with the keep-alive flag). Having multiple "instances" could be triggering the edge case that these 2 API's only accept a total outstanding (unsent by the browser) page of 64Kb, so the more events / instances then the more likely this situation could occur. If both of these cases on the network tab the "Type" will show as ping, if they are showing as xhr or fetch then they are the normal API's and therefore this is not this situation and more likely multiple instances receiving and sending (apparent) duplicate events.

from applicationinsights-js.

Thanks for the help!

I think I just needed the namePrefix to be set to a different value for each component. I didn't use that in V2, so maybe that's new, or required now?

I no longer see the duplicate tracks in my network tab, just by setting the namePrefix (I didn't have to use the other two options). Nothing bad happens when I switch browser tabs either.

I see that my two test components now both have their own space in SessionStorage:

For your information, I did see the option onunloadDisableFetch but I didn't see the option disableSendBeaconSplit. These are the only two 'beacon' settings I see:

By the way, I'm not using the automatic exception tracking, just the appInsights.trackException function. Still not sure why those have dupes as well, but maybe this will be solved with the upgrade to 3.0.7 AND setting the namePrefix...

One last thing: why don't you set the namePrefix to something unique upon instantiation automatically? Might be something to look into, although there might be good reasons not to do that, too...

Thanks so far, I'll keep you posted if anything weird happens!

from applicationinsights-js.