getsentry / .github Goto Github PK

@bruno-garcia:

4 weeks is nothing
I'd go with 6 months :D

@benvinegar:

4 weeks seems aggressive - is that typical for a project like this?
I was thinking a few months
Eg it tagged a ticket of mine that was 2 years old - that makes sense

When an external user comments on an issue in Status: Needs More Information, we should switch back to Status: Untriaged.

The only repos that are shared by multiple teams are sentry and sentry-docs. The vast majority of our repos belong to only one team, so any tickets opened there are already routed. They are not triaged, however, and we do want global visibility into triage, so we need to standardize the process across all repos.

When we route a ticket, find a way to send a Slack message.

getsentry/sentry-docs#3093

getsentry/sentry#23681

This line now emits output when it didn't before, resulting in garbage like at getsentry/sentry#31735 (comment) (pre-edit).

getsentry/sentry#24043

https://github.com/getsentry/onpremise/issues/837

Hi. I did use the template but added an additional headline. This or something else might have tripped the parser. I added "Feature Request: " to the title after the fact, wasn't sure if it was needed.

Thanks for taking a look!

Would be great when looking at a Jira ticket that has a public GitHub shadow ticket, to see the GitHub engagement level (comments, reactions) at a glance in Jira (sidebar?).

#49

We might already have this, put it in place if not.

e.g.:

For invalid issues we comment, but for valid issues we do nothing. Let's add a welcome message that gives us a chance to:

• link to the processing process and emphasize the SLAs
• nudge paying customers towards support
• ...?

Reticketed from getsentry/open.sentry.io#1.

getsentry/sentry#27617

Environment

GitHub Actions

Steps to Reproduce

1. Set up a repo with an issue template with no headers.
2. As a non-org-member, create a new issue with no headers in that repo.

Expected Result

Issue is allowed, since there is a template with no headers.

Actual Result

Issue is not allowed.

Parent: #22

Problem

Not every repo needs to have its own issue templates defined. In fact, it's probably true that almost all of our repos should be using the global default templates. The issue validator bot/action as it stands today only checks new issues against templates defined in the local repo.

Solution

We should improve the issue validator bot to check local templates if they exist, and global templates otherwise.

There are already a few instances of people using LLMs to generate very long, detailed, and wrong solutions to peoples' raised bugs
getsentry/sentry-javascript#8285 (comment)
getsentry/sentry-javascript#8281 (comment)

It's not immediately obvious that those were generated by LLM (even though it's quite glaring in hindsight), and it's quite exhausting to respond in a way that don't hurt their feelings while still communicating to anyone else (mainly the maintainers) who might just be skimming through that the large blob of confident-looking hallucination-masquerading-as-a-solution is actually not a solution to the problem.

Thoughts on adding something to Code of Conduct that no responses should be generated via LLMs without proper disclaimers for the portions that are generated? And ideally also no LLM generated responses should be posted without the poster verifying that they really tried to understand both the originally posted problem and has done their best to verify the veracity of the generated text?

(prior art https://news.ycombinator.com/item?id=36178342)

We're closing too many issues as invalid. We have a bunch but I don't see another issue to track this already.

Latest: getsentry/sentry#24526

Part of #8.

@BYK at getsentry/self-hosted#830 (comment):

we may want to make the issue validation bot a bit less aggressive. I think it tripped on the extra heading @/JeSappelleRoot added.

Problem

We are like 1990s PHP developers right now. We have no development pipeline, no tests, we're copying and pasting files all over the place, we're editing live on the server. How can we bring normal software development practices to this strange new serverless GHA world?

Solution

I want to store workflows in .github and deploy from there out to target repos, in phases (i.e., onpremise, then sentry-docs, then sentry, then ... ?). I envision an action here in .github to do the deploy, with target_repo as a parameter (probably required, with a special case of all for GA deploy ... or maybe repo groups are defined in a yaml or something).

That's deploy. CI? I want to make a PR against this .github repo—as I've done here—and have CI kick in that runs the tests, but here's the catch: they need to run in a cleanroom repo, we don't want test fixture to clutter up this repo. I made a .github-ci repo to use for that, but now I think we probably actually want to create repos on the fly so that multiple CI runs in parallel do not clobber each other. I envision an action here in .github to do that, wired up like normal CI in any other repo (e.g.).

Okay! Deploy and CI! How about dev? I made a .github-chadwhitacre repo for myself that has .github as an upstream, so I can stay in sync and do my dev work off of the latest. The best way to develop GitHub Actions is by running them in a true GitHub Actions environment, that is to say, in the true actual cloud. ☁️ Trying to use nektos/act is going to be too heavy and drifty by comparison. 🚢 I made a helper script called ons (for "on save" ... also considering gah as a play on GitHub Actions and the exclamation "Gah!!") that commits, pushes, and kicks off test actions in my dev repo. I've been running it manually but it could maybe be hooked up to run on save in one's editor if so desired. My workflow is:

1. Edit files locally.
2. Save.
3. ons
4. Go to browser tab with dev repo "Actions" open, refresh.
5. Click, click to current test run.
6. Inspect output.
7. Repeat.

Test actions are named foo-test.yml (à la foo_test.go).

Here's a wrinkle: I want to keep test yamls out of the main .github repo, just like we don't ship test code to prod in other software architectures. Actually, nevermind: .github is not the prod environment, it's the final step before prod. The action that deploys action yamls to the actual target repos will not ship test yamls. Done!

Todo

• Write CI for .github.
• Write an action deploy action (starter script).
• Clean up and publish ons.
• #30
• #31

The global issue templates in this repo aren't working because the implementation is weird and we need to nest them in .github directory for them to work:

https://github.community/t/issue-template-for-organization-does-not-work/2049/17

h/t @bruno-garcia

Example getsentry/sentry#23793.

Not sure how much we can do about this. Maybe we can ignore transferred issues if that information is available in the payload?

getsentry/sentry#25150

Reticketed from https://github.com/getsentry/.github/pull/21/files#r563331189.

Problem

The Open Source Team is responsible for routing all incoming issues to the appropriate team, but we don't have a good queue to work against, which makes it hard to ensure that we're not missing issues. The general GitHub notification stream is waaaaay too noisy (regardless of how it's consumed: email, web, etc.).

Solution

Teach the validate-new-issue action to apply a Status: Unrouted label to issues that pass validation. With this label humans (for now) can use a search like this to stay on top of routing:

https://github.com/search?q=org%3Agetsentry+label%3A%22Status%3A+Unrouted%22+state%3Aopen&type=Issues

getsentry/sentry-docs#3150

(just re-formatted to follow your template, please re-open the issue. Thanks)

Routing is currently a multi-click process:

Routing in the sentry and sentry-docs repos is done with Team: Foo labels and a comment mentioning @getsentry/team, with the "Team" field in the Registry being the source of truth for who owns what.

It could pretty easily be reduced to a single click: applying a Team label.

Parent: #22.

Acceptance Criteria

1. stalebot is configured in this repo .github
2. stalebot operates on all repos
   1. could be current GitHub Action + tooling to propagate config to N repos, or ...
   2. probot? cf. #5
3. exempt-issue-labels includes additional labels in order to fit in with existing triage workflows (cc: @bruno-garcia).

Auto close issues and PRs not adhering to the templates

Example: https://github.com/seferov/pr-lint-action

Todo

• better log message than full echo template
• auto-close on fail, with comment
• write real comment
• deploy to onpremise
• close hole: getsentry/self-hosted#822
• look into \r bug: #8 (comment)
• relax match constraint: #15
• harden against additional ISSUE_TEMPLATE situations: #26
• make log download more robust (seeing off-by-ones, probably race condition?) → #30
• probably convert to an action
• deploy to onpremise
• deploy to sentry-docs
• deploy to sentry

Validation bot is supposed to skip all getsentry org members, public or private. Seems to be whiffing on private members. Is the right API out there? Any hope for the flowers? Maybe give the bot more perms?

If we can't fix this, consider:

• improving the log message
• adding a note to the comment
• notifying engineering team
• incorporating into onboarding

This would make queries cleaner.

Follow-on from #12, from @dcramer in Slack:

ideally we can make them understand what discord is for as part of this
also we should setup a short URL for this somehow
e.g. like sentry.gg/coc

From @jan-auer in DM:

I quite dislike the lag that GH actions come with for bots. A few examples:

• Our danger actions come with 20-30s delay of editing PR descriptions.
• The new stale bot doesn't immediately remove the label once there's activity (at least I haven't seen it do that).

Suggested solution is Probot.

If there's already a Status: * label on an issue then the Team: * action should not re-route.

E.g.

https://github.com/getsentry/sentry/issues/26121

Environment

GHA test suite (run manually atm).

Steps to Reproduce

1. Run test suite for validating new issues in GHA.

Expected Result

Test suite is consistently green.

Actual Result

Test suite is occasionally red with an error like this:

download-and-extract backed right off the edge!

and a mismatch in the expected and actual issue number in the downloaded log file.

Notes

We make an assumption that run_number == issue_number - 1 (offset vs. ordinal), which holds most of the time. We make this assumption because there doesn't seem to be a way to request the logs for a workflow run based directly on the issue number for the issue event that triggered the run, and it's going to be less download-y to start with run_number and radiate outwards in the list of all recent workflow runs than some other strategy (such as always starting with the most recent run and working backwards). The way I see to resolve this bug is to implement the radiating outwards. ☺️ But! I'm open to other solutions ... best would be a way to determine with certainty from the list of runs which run is the one we want.

• issue template with no headers: #29
• issue template from .github repo → #31
• ISSUE_TEMPLATE.md not relevant at the moment, not sure it ever will be
• etc.? I think we're good.
• issue template exact match (e.g.)
• #25

We just turned on SSO for our GitHub org, need to audit bots and make sure they still work.

Relevant: https://github.com/actions/labeler

Mirror: https://app.asana.com/0/1169344595888357/1198193177012935.