Comments (5)
Monitoring tooling I've looked at: https://github.com/MeasureOSS/Measure
from zalando.github.io.
This issue needs to be done first, in order to get started on evaluating monitoring tools. #26
from zalando.github.io.
Purpose
To define what we want to measure, how should the monitoring dashboard look like, and the type of reports and alerts to be generated from the monitoring infrastructure MVP.
Problem Statement
The Open Source team is a newly formed entity within Zalando. It is very important we gather metrics around Zalando's Open Source involvement so that we can measure the impact this new team is able to bring and be able to effectively drive the open source strategy that the team has in place.
Background information
CatWatch is a Zalando Incubator web application that fetches GitHub statistics about the total number of repositories, contributors, languages, forks, stars etc. for our GitHub accounts, saves this data in a database, then makes the data available via a REST API.
This data is limited and we will need more insights (discussed below). In order to achieve this, below is the proposed solution.
Proposed Solution
Points italicised are out of scope from the MVP.
Measure
Analytics, dashboard and reports: Be able to measure all the activity in Zalando's Github organisations. This would be presented in a customised dashboard view and reports generated periodically.
- New projects added to the incubator
- No. of projects graduated from the incubator
- Total no. of PRs opened vs merged
- Time taken to merge a PR
- PRs from external contributors
- Total no. of issues created vs closed
- Time taken to respond to an issue
- Total no. of comments in an issue
- Issues created from external contributors
- Issue comments from external contributors
- Top contributors - weekly, monthly
- Organisation wide
- Project specific
- External top contributors
- No. of new contributors
- Organisation wide
- Project specific
- External new contributors
- No. of external contributors
- Organisation wide
- Project specific
- Female contributors
- Comparison of all the above metrics with old projects vs new projects
- No. of cross-team collaborations (inside Zalando) - Issue + PR
- No. of new collaborations - Issue + PR
- No. of female participants - Issue + PR
- No. of non-tech collaborations - Issue + PR
- No. of PRs opened
- Time taken to open vs merge cross-team PR
- No. of issues created
- Time take to open vs close issue
- No. of Zalandos contributing to external projects
- No. of issues created to external projects
- No. of PRs opened to external projects
- Time taken to open vs merge Pr
- Time taken to open vs create issue
- No. of issue comments on external projects
- No. of female contributions to external projects
- Issues created by female contributors
- PRs opened by female contributors
- Issue comments by female contributors
- No. of projects with releases
- No. of new releases
- Diversity(Over time, these numbers should improve)
👩 - No. of female contributors
- No. of female maintainers
- No. of new female contributors
- No. of top female contributors
- No. of external vs internal female contributors
- No. of PRs opened vs merged by female contributors
- No. of issues opened vs merged by female contributors
- No. of issue comments by female contributors
- No. of non-tech contributions
- No. of PRs opened to update docs
- Time taken to open vs merge
- Issues opened to updated docs
- Time take to open vs close
- No. of external contributors
- No. of female contributors
- Retention / Attrition
- No. of external contributors hired
- Time taken to hire
- No. of internal contributors who no longer work for Zalando
- Time spent contributing to open source by internal contributor before leaving
- No. of external female contributors hired
- No. of internal female contributors left
- Compliance
- Number of compliant projects based on Rules Of Play
- No. of new PRs which aren't compliant (Over time, these numbers should decrease)
- Social media
- Twitter impressions on a tweet mentioning particular project
- Twitter impressions from top contributors published
- Increase in twitter followers due to the above 2 points
Alerts
- New project added to the incubator
- Project graduated from the incubator
- New PR opened
- New issue created
- Organisation stats reaching a milestone
- Stars, contributors, forks == 100, 500, 1000 etc.
- Project stats reaching a milestone
- Stars, contributors, forks == 100, 500, 1000 etc.
- PRs, issues reaching a milestone = 100, 200, 300 etc.
- PR, issue staying open for a long time, more than a month
- PR, issue unattended for more than two weeks
- External contributor hired
- Internal contributor left
- New PR, issue from a female contributor
- New tweet about a Zalando specific project
from zalando.github.io.
MVP
Measure
Domain | Feature |
---|---|
Project | No. of new projects added to the incubator |
Project | No. of projects graduated from the incubator |
Project | No. of projects with releases |
Project | No. of new releases |
Domain | Feature |
---|---|
PR | Total no. of PRs opened vs merged |
PR | Time taken to merge a PR |
PR | No. of PRs from external contributors |
Domain | Feature |
---|---|
Issue | Total no. of issues created vs closed |
Issue | Time taken to respond to an issue |
Issue | No. of issues created from external contributors |
Domain | Feature |
---|---|
Contributor | No. of new contributors |
Contributor | No. of external new contributors |
Contributor | Top contributor |
Contributor | Top external contributor |
Domain | Feature |
---|---|
External participation | No. of Zalandos contributing to external projects |
External participation | No. of issues created to external projects |
External participation | No. of PRs opened to external projects |
Alerts
- Organisation stats reaching a milestone
- Stars, contributors, forks == 100, 500, 1000 etc.
- Project stats reaching a milestone
- Stars, contributors, forks == 100, 500, 1000 etc.
- Total PRs, issues reaching a milestone == 100, 200, 300 etc.
- PR, issue staying open for a long time, more than a month
- PR, issue unattended for more than two weeks
from zalando.github.io.
As an additional KPI we should measure our teams ability to publish content on the team blog, social media and participating in events
- Number of blog posts on the tech blog
- Number of talks from a team member
- Twitter impressions on Zalando Github repo links
from zalando.github.io.
Related Issues (20)
- Productivity Indicators HOT 1
- Open Source License Scanning HOT 4
- Order Software for e-learning
- Prepare for TechSUP reboot
- Guide to publish blog on the contribution file
- January Review
- Open Source Repo Leak Scanning POC
- Add PR review checklist HOT 2
- bug: blog/page1 gives 404 not found HOT 1
- 404 on zappr link HOT 2
- CLA whitelist is missing Lightbend CLA HOT 1
- Remove mention of ZMON
- Mention requirements to join Zalando organization
- First link in the blog pagination is broken
- Out of date Gemfile.lock contains dependencies that can't be installed
- Typos in PR and issue templates HOT 2
- Allowlist Meta CLA HOT 2
- Not Added Copyright Notice at bottom of a site HOT 1
- Add Scala CLA to the Allowlist
- License links lead to 404
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zalando.github.io.