Retention for FxA-attached services about fxa HOT 8 CLOSED

permanently stop using

Yesterday @ckarlof shared with me a very nuanced definition of "churn" that doesn't take into account any idea of "permanence", we may need to tweak and clarify this a little.

from fxa.

Let's start with a very simple model for churn. I propose we instead focus on retention, because it's simpler to talk about. I know if a user stayed. It's harder to know if user went (for good). Below is what I'd like to see for FxA attached services. I got it from here. The model is simple:

"The first graph shows a retention curve: The number of days that have passed since the initial install, and what % of those users are active on that particular day. As my readers know, this is often used in a sentence like “the D7 retention is 40%” meaning that seven days after the initial install, 40% of those users was active on that specific day."

Replace "initial install" with "first use of a service".

After we have the ability to create this graph for a set of users, we can start asking questions like:

• How do the retention graphs compare for single vs multi device users?
• How do the retention graphs compare for high volume sync users vs light volume sync users?
• How do the retention graphs compare for single vs multi service users?
• How do the retention graphs compare for users that added an avatar image within the first 3 days vs those that didn't?

We also might consider including other early events on the x-axis, e.g., "verified" account.

from fxa.

flipping churn and retention graphs is fine.

If we want a dashboard as a method of measuring against A/B testing. We can compare the 7 or 14 day retention of users A and B. If there's a decrease in retention, then we know the experiment has negative affects on our core KPI

from fxa.

If there's a decrease in retention, then we know the experiment has negative affects on our core KPI

That's the idea.

from fxa.

@ckarlof Capital idea, obviously. I prefixed several of the retention-related in Datadog with R:

Would love plus the ability to see how variants of the registration/onboarding flow perform in the longer term.

from fxa.

Bugzilla bug on Sync retention (Moz employees only): https://bugzilla.mozilla.org/show_bug.cgi?id=1173407

from fxa.

(I changed the bug title to reflect what we actually plan to measure)

from fxa.

Exporting these FEATURE cards from Aha has not turned out to be useful in practise, I'm closing them in favour of more direct use of Aha.

This feature lives at: https://mozilla.aha.io/features/FXA-24

(And I feel like we've socialized the Retention Graph idea above enough now that we're not losing any context by closing this out)

from fxa.