Comments (9)
No, email integration is not worth the effort because we are in the habit of checking Bayes regularly.
from aqua.
Yes, email integration would still be valuable and we should take the time to implement it.
from aqua.
@jonathanolson made a compelling argument, I'm going to move my thumb's up to the "not now" column. Once things are very stable and errors are more rare, we can re-evaluate.
from aqua.
We are in the habit of checking Bayes regularly, do we need to add an email component too? I'll create voting by thumbs ups: You can unassign yourself after you vote.
from aqua.
phetsims/chipper#410 (comment)
Agreed if new bugs are emailed.
from aqua.
I see a lot of "yays" for this. I'm interested in what the "new bugs" would really mean then.
For instance, if I cause a runtime error that hits every simulation on startup, that will cause failures in ~87 runnables, each currently with 8 tests (~700 individual tests that were passing will start to fail).
Furthermore, the same test can fail with many different stack traces (depending on the fuzz).
We also have flaky tests (ones that fail randomly mostly based on the fuzz events - once a day/week/month).
If we have a system that will give us useful information on failures (but won't spam us with 1000s of emails - or even more than 1 email per type of failure), then that sounds great (and non-trivial).
No, email integration is not worth the effort because we are in the habit of checking Bayes regularly.
Presumably we don't need to check bayes regularly because (a) we are almost always linting/testing before pushing, (b) usually for commits that would have the potential to break things, we'll push, wait and check, and (c) QA is checking bayes regularly, to catch things that weren't anticipated above. I'm not even checking bayes weekly, but usually only if I need to do some testing that requires knowing about currently failures (or branching a sim from master), or if I notice something broken locally and want to quickly confirm if it's an issue for others.
So I'm fine if someone wants to work at email notification, but it doesn't currently seem like it's worth the effort to me.
from aqua.
I voted no, and thought I should explain myself. I'm in the habit of checking Bayes every morning, and if I see anything that needs attending to, I put it on my to-do list for the day and prioritize it relative to everything else. If I change something in common code, I set a timer and check Bayes some time later. Email would be unlikely to change this behavior for me, but I wouldn't object to it being added if others would find it valuable.
from aqua.
I'm changing my "yay" to a "nay". I think it would be difficult to identify "new issues", and I don't want to hear about chronic issues repeatedly. And it sounds like people are already successfully identifying problems manually. So I don't think this is worth the work to automate this. Perhaps better to emphasize that, if you make a common code change, then you'd better have a look at bayes after you commit and push.
from aqua.
Based on preceding remarks, I'm inclined to close the issue.
from aqua.
Related Issues (20)
- Report all new CTQ errors on Slack HOT 10
- Can CT fuzzing report which screen was active when a fuzz error occurs? HOT 1
- Create a CT client based on Puppeteer. HOT 15
- CTQ error opening puppeteer HOT 6
- Puppeteer Chrome clients fill up /tmp on bayes HOT 5
- Improve continuous server error handling HOT 3
- CTQ throw an error if you click on a red box HOT 1
- CT has a number of grey cells and tests seem slow/not being done HOT 2
- MaxListenersExceededWarning with puppeteer Client. HOT 1
- CT: Error: Failed to launch the browser process! HOT 25
- Add dedicated CT server HOT 26
- Improve CTQ notifier HOT 6
- Local aqua doesn't respect sourcemaps HOT 2
- PM2: Should we add log rotation to bayes HOT 18
- CT malformed invocation failed: string instead of callback HOT 2
- Increase browser coverage on automated tests
- CT Lint failed status code 1 HOT 6
- CT isn't finishing columns HOT 3
- Why is CT doing `&fuzz&stringTest=xss` ? HOT 6
- Add a single phet-io API check to quick CT HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aqua.