Web application for viewing Glean debug pings in real-time.

Since this repository is private, CircleCI is not able to build from forks. As a workaround PRs should be submitted from branches pushed to origin.

Required node version: 14.20.1

If you just cloned this repository, first you should install dependencies:

npm install
cd functions && npm install && cd ..

Run the following command to generate required Glean files (needs re-ran anytime Glean .yaml files are updated):

npm run build:glean

Run React application locally:

npm start

Application will start connected to development project. For details, including building and deployment, see Deployment.

Send debug ping to GCP ingestion:

curl --header 'X-Debug-ID: test-debug-id' -XPOST https://incoming.telemetry.mozilla.org/submit/glean/events/1/$(uuidgen) -d '{"$schema":"moz://mozilla.org/schemas/glean/ping/1","client_info":{"app_build":"59f330e5","app_display_version":"1.0.0","architecture":"arm","client_id":"6ff20eb7-e80d-4452-b45f-2ea7e63547aa","device_manufacturer":"Mozilla","device_model":"phone123","first_run_date":"2018-10-23-04:25","os":"Android","os_version":"3.2.1","telemetry_sdk_build":"abcdabcd"},"ping_info":{"ping_type":"full","seq":1,"start_time":"2018-10-23 11:23:15-04:00","end_time":"2018-10-23 11:23:15-04:25","experiments":{"experiment1":{"branch":"branch_a"},"experiment2":{"branch":"branch_b","extra":{"type":"experiment_type"}}}},"events":[{"timestamp":123456789,"category":"examples","name":"event_example","extra":{"metadata1":"extra","metadata2":"more_extra"}},{"timestamp":123456791,"category":"examples","name":"event_example"}]}'

Clients are submitting pings to Mozilla telemetry ingestion system. Pings tagged with X-Debug-ID header (e.g. X-Debug-ID: test-name) are picked up by debugPing function and loaded to Firestore.

Frontend is implemented with React hosted on Firebase, it leverages Firestore's real-time query update capability.

Pings from structured-decoded-debug and structured-error topics are pushed to HTTP Functions via Push Subscriptions. We can't use PubSub-triggered function here because it doesn't support cross-project access.

For more information about the functions, see functions/README.md.

Glean debug pings with some fields extracted are stored in Firestore. There are two collections with flat documents used:

clients
+ key: string // {client_id}_{debug_id}
+ appName: string
+ debugId: string // value of the `X-Debug-ID` header
+ geo: string // city and country of the last submitted ping, from GeoIP
+ lastActive: string // timestamp of the last ping submission
+ os: string

pings
+ key: string // document_id
+ addedAt: string // submission timestamp
+ debugId: string
+ payload: string // submitted ping payload
+ pingType: string // document_type

Ping collection is indexed.

Pings are retained for a limited time. Every day, a function that deletes pings and client entries modified earlier than 21 days ago is triggered via Cloud Scheduler.

Views are leveraging Firestore's realtime query updates.

Simple Google Signin-based authentication is inplace. It allows to log in with any Google account, although only those under mozilla.com domain are allowed to fetch data from Firestore.

This project uses Glean.js SDK for data collection. If you want to disable data collection, please turn on browser's Do Not Track feature.