Expand a set of tests ensuring we don't capture any text or images when config disallows it, or when privacy option is set on a node and its subtree. Because we value our users' privacy, our text/image masking tests must be really thorough.

Are there any experiments to show the impact on web page performance due to this script? Like does it increase the long tasks (tasks > 50ms) etc.

Currently we only strip 'src' attribute on images, if config tells us to hide images. We should improve by also masking images that come from CSS and other possible sources. Keep in mind that CSS might be a referenced resource.

Compression workers started timing out in Chrome Headless. And in general, headless browser tests behavior became quite flaky. We need to fix this ASAP.

Currently, each payload's envelope contains redundant info such as userid and page url. We can send all of this fixed data as an impression metadata property with only the first envelope.

Currently we can't arrange impressions within sessions in the right order relying purely on Clarity data, because all timestamps in envelopes are page-based, so they can't be compared in global time.

This is an interesting project. But the current documentation on getting started is not so great. For example, these are some of things i had trouble with

1. How to use this library ? Simply (gulp) build and get the clarity.js file and include that in my website ? Should it go in head/ end of body ?
2. When does the library get activated/start tracking ? Do i need to explicitly call the clarity.start() method ? or something else ?

I ran across two issues, the first is more of a use error so I'll include the solution I came up with for that as well.

Elements not observed

Firstly, no Input/Textarea's were being observed.

What I did during tests:

• Added clarity script to top
• Used clarity config:

var config = {
        plugins: ["viewport", "layout", "pointer", "performance", "errors"],
        uploadUrl: "http://localhost:6969/telemetry/test", // web server that processes the payloads
        projectId: "test",
        urlBlacklist: [],
        delay: 500,
        batchLimit: 100 * 1024,
        totalLimit: 20 * 1024 * 1024,
        reUploadLimit: 1,
        showText: true,
        showLinks: true,
        showImages: true,
        sensitiveAttributes: [],
        timeToYield: 50,
        instrument: false,
        cssRules: false,
        uploadHandler: null,
        customInstrumentation: null,
        debug: true,
        validateConsistency: false,
        backgroundMode: false
    };

• opened webpage, did clarity.start() in console
• moved around mouse, scrolled up and down, typed some stuff in input, submitted some forms, continued scrolling
• there are no iframes, embeds, or contenteditable divs in the webpage, just vanilla input elements and forms

Debugging

Did not see the input changes pop up anywhere on server. I modified Layout.prototype.watch to include the following sanity check:

// added toUpperCase to make sure string is uppercase
if (element.tagName.toUpperCase() === "INPUT" || element.tagName.toUpperCase() === "SELECT") {
    console.log("FOUND AN INPUT ELEMENT!!!!!"); // sanity check
    bind(element, "change", this.layoutHandler.bind(this, element, 3));
    this.watchList[layoutState.index] = true;
}

This did not trigger anything. For further sanity:

Layout.prototype.watch = function (node, nodeLayoutState) {
    if (node.nodeType !== Node.ELEMENT_NODE || this.watchList[nodeLayoutState.index]) {
        return;
    }
    console.log(node.tagName); // second sanity check
    ...

No input tags were logged. Furthermore, the only tags logged were SVG nodes (path, circle, g, etc.) and div (not even p tags logged).

Solution

The solution is quite simple, put the call to clarity.start() in the head somewhere. This solved the issue as it now observes the entire DOM tree.

I think the expected behaviour is that it would traverse and observe the existing DOM tree on start anyways, but have not come up with a modification to allow this currently.

Text still masked even with showText: true

What I did during tests:

Same as above except clarity.start() at the end of clarity.js (not using minified to allow debugging).

(inl. showText: true and debug: true)

Debugging

Put breakpoints before config.showText was being used, and did a sanity check in console to make sure it was true (it was).

Workaround

I changed each instance that config.showText was being used (usually var showText = ... config.showText ...;) to have it as true, usually:

var showText = true; //... config.showText ...

This seems to have fixed the issue. I believe the issue may lie with the forceMask variable, perhaps this is intended behaviour and I am not understanding how forceMask is calculated?

Any help would be appreciated. Thanks!

Currently we use strings to identify types

We have a git hook, which runs tslint on the project before a commit can be made to make sure that code standards are passing. If a staged version of the file fails tslint check, but an unstaged version passes, then pre-commit tslint check would pass, but the failing version would get commited. Need to upgrade hook to inspect staged versions of files.

Clarity currently can't handle reading the state of canvas elements on the page and isn't wired up to see when they change. Similar products punt on canvas support:

HotJar does not support or according to their documentation:

https://help.hotjar.com/hc/en-us/articles/115011822728-Troubleshooting-FAQs-for-Recordings

Why do some elements of the page appear blank or without content?
There are a few limitations on the type of content Hotjar supports for Recordings. If you've noticed that your Recording appears blank or seems to have some missing content, here are a few possibilities:
• Your page has a canvas/SVG element
• Your page uses Adobe Flash to render the content

Similarly for FullStory: https://help.fullstory.com/using/google-maps-during-playback https://help.fullstory.com/using/148817
• SmartLook (https://www.smartlook.com/help/smartlook-user-guide/)

We would have to do support “smart” capture if we didn’t do a polling based approach of finding all canvas element on the page and polling there getImageData() every time we poll. The interception approach with polling could be combined – intercept to find out when a canvas is changed and then (after some delay) trigger a capture of the image data.

https://github.com/Rob--W/canvas-interceptor/blob/master/canvas-interceptor.js

Give users opportunity to create custom instrumentation events through Clarity. This could be implemented by adding .instrument(key, payload) api to the clarity object, which would add an instrumentation event to core, containing key and payload in its state.

Include some metadata about the page in the 'Teardown' event. This data should probably indicate the quality of the impression (e.g. how many XHR posts failed, whether ShadowDom was always consistent, JS error count, etc).

Use app-wide event dispatchers to reduce coupling in communication between various app components

I attempted to create an account and my websites were not approved for several months now.

The email address has not responded either.

Have you thought about using some kind of homomorphic encryption for the submitted data, for example https://github.com/mhe/jspaillier, instead of masking input?
This would make it possible to learn on encrypted data from the users and would not be as concerning as masking content with a blacklist approach (which obviously leaks private data see #81.)

Add functionality to observe CheckBox ticks and RadioButton selections