palantir / bulldozer Goto Github PK

The README includes a sample configuration, but does not describe what the behavior of any of the settings are. (This prevented us on Go Cloud from considering Bulldozer for our project, see google/go-cloud#687.) Things in particular we'd like to understand are:

• Does this consider branch protections and code review approvals?
• Does this wait for CI to catch up?
• What happens if the original author adds more commits after the last review? Does that get merged in or does the bot stop?

In order to allow bulldozer to merge, I can not check the Restrict who can push to matching branches in the settings -> protected branch

How can I allow only bulldozer to merge PR's and not everyone with "write" permissions on the repo?

Is there any way to create some user called "bulldozer" and give these details to the bot?

Currently, Bulldozer's "update me" will continuously update a PR every single time that a commit is added to the base branch.

I suggest that we change the behaviour of "update me" to only update the PR once. The idea is that once the update label is applied, bulldozer would update the PR and subsequently remove the label.

I think that this change is behaviour is desirable because we have seen the previous behaviour result in unexpected load to GH/CI and interact negatively with other automation (excavator)

Hi,

Thanks for releasing this, it's almost exactly what I need. One workflow I have seems to require a workaround, which is whitelisting/blacklisting based on the target branch of the PR. For context, we want PR's into the production environment not be to squashed, but all other PR's to be.

I've dealt with this by whitelisting labels and only applying the label to branches that should be dealt with, but it would be helpful to be able to whitelist target branches OR if it was possible to apply merge strategies based on the target branch?

I think another workaround I could apply would be to run two bulldozer apps side-by-side, but that's also a pretty big burden.

If it's something you're open to, I'd be happy to take a shot at developing the feature.

Either way, thanks for the tool 👍

On some repositories, after Bulldozer merges a PR, GitHub Enterprise 2.14.6 still sees the state of the PR as open: it shows up in open PR lists and has a button to close it. The merge is complete and the commits exist in the target branch, so it is only the PR state that is incorrect.

We have an open support issue with GitHub for this, since it seems like an internal race condition or an issue with app merges, rather than a bug with Bulldozer. The 0.4.x OAuth app never had this problem despite using very similar merge code.

If we can't figure out what's wrong, we'll probably need to add a workaround to close PRs via the API after they are merged.

Steps to reproduce:

1. Create a pr from foo -> master
2. Push a commit to master
3. Add the UPDATE ME label to your pr

Expected outcome: bulldozer updates the pr
Actual outcome: no action is taken

Similar to policy-bot and Probot apps, Bulldozer should provide a user-friendly landing page when people visit the server root URL. It should include:

• A brief description of the bot
• A link to the install page on GitHub
• Steps for getting started with a minimal configuration
• The current server version

Bulldozer is configured via labeled PRs, but a human has to create all those labels by hand. If I enable bulldozer on a repo which does not have the labels, they should be created for me.

On one of our internal repositories, people submit changes from forks. These forks are sometimes out-of-date and are brought up-to-date by a series of merge commits. While GitHub correctly detects which files have changes, when Bulldozer squash-merges the PR, the resulting commit message can have hundreds of lines, including references to just about every commit that was included via one of the merges.

This makes history look pretty weird and can also lead to things like people getting mention emails from commits that merged months ago.

As a reviewer on the repo, it's unclear to me that this will happen before I add my approval, so it would be nice if Bulldozer could protect me from this result.

Some options:

1. Add a configuration option to always use the PR title and either an empty message body or the body of the PR.
2. Place some character limit on the size of the commit message; I don't know if GitHub exposes the generated message in the API
3. Use the same method as GitHub to generate commit messages (a list containing each member commit), but ignore all merge commits

What happened

Someone just asked why bulldozer wasn't working on a PR and all the configuration looked right (bot enabled, correctly configured etc). Turns out it just hadn't delivered any webhooks.

What should have happened

When the bot is first enabled, it should crawl open PRs and merge ones that match the criteria already

#50

• #50 (comment)
• #50 (comment)
• #50 (comment)
• #50 (comment)
• #50 (comment)

Hi. First, thanks for you all developers of this great tool.

I want pattern match for status checks which are defined in the required_statuses of .bulldozer.yaml because some CI status checks cannot be statically defined. For example, GitLab CI injects target branch names into their statuses (e.g. ci/gitlab/<branch-name>).

The solution will be a small change in evaluate.go where simple key existence check is used currently.

Can I hear your opinions?

I just tried to run bulldozer and I'm getting the following or similar errors when receiving events from Github:

Get https://api.github.com/repos/celo-org/celo-monorepo/pulls?per_page=100&state=open: could not refresh installation id 524951's token: received non 2xx response status "404 Not Found" when fetching https://api.github.com/installations/524951/access_tokens

The installation ID is correct, so I'm not sure what the difficulty is, and I'm quite new to Golang. Are there any points for debugging this?

I've recently seen a large number of PRs opened by excavator remain open although they are green and have the merge when ready label applied. Toggling the label or interacting with the PR in any way seem to trigger the merge

Why exist vendor?, dep should to install it...

If a PR is created whose base branch has no branch protection rules, then bulldozer will immediately merge that PR.

This seems undesirable. Opening a PR with a non-default base branch is a common workflow when one PR depends on another currently open PR.

Say you have two branches A and B, each with a PR up.
The PR for A merges into develop, the PR for B merges into A.

Currently, if deleteAfterMerge is set, A gets deleted when merging and B gets automatically closed.

It would be great if deleteAfterMerge checked first that no other PR depends on A.

Currently, we need to add a config file (eg .bulldozer.yml) to each repo.

I've add support for global configuration as repo fallback configuration.

We can install bulldozer as GitHub app only to projects need to enable this feature.

You can view my implementation here: https://github.com/richard1122/bulldozer/commit/ee5a0b79d0855b83821cadec3eb5a73c31b05b36

If this feature is acceptable, I 'd like to send a PR.

Awesome App, thanks!

I could find a way to set the options based on the author of the PR. Is there an option to set it?

BTW: it would be also very useful to set not only the author but to set if the author belongs to a team. E.g. merge the PR only if the author has commit rights on the repo (AKA belongs to a specific team).

Thanks!

I have set-up bulldozed using docker. The command being used to run the docker container is:

docker run -it -d --restart unless-stopped -v /path/to/bulldozer.yml:/secrets/bulldozer.yml -p 8400:8400 --name bulldozer palantirtechnologies/bulldozer:latest

Nginx reverse proxy has been set-up on xyz.example.com which routes traffic to the docker container running on http://localhost:8400

When I try to access the http://xyz.example.com , I am getting a 404 from the container. When I am trying to hit http://xyz.example.com/api/github/hook , I am getting a 404 again.

Can someone explain what I am doing wrong here?

https://github.com/palantir/spark

I'd love to get the MERGE WHEN READY functionality on the public github repos I maintain, like palantir/spark above.

Is this something we could do?

Seems like a much more natural fit to most GH PR workflows.

Currently, when the update me label exists on a pr, the pr is only updated when the target branch is updated.

However, it might be the case (in fact it's likely) that the pr is out of date at the time the update me label is added, in which case it seems worth it to update the branch then.

Say I have two reviews:

• 1 approval
• 1 request changes

If I have Merge When Ready on my PR, it Bulldozer will merge. Can we merge only if there are no Request Changes reviews in place?

Is there any current workarounds to use Bulldozer when Restrict who can push to matching branches is enabled on GitHub?

From looking at other projects, it appears it's currently impossible as GitHub does not provide a way to give push permission to the Bulldozer GitHub App directly, only user accounts or groups can be selected.

Some projects have worked around this by having a 'real' user account for the bot to use in this situation.

• Same issue in Renovate: renovatebot/renovate#846
• Similar issue in dependabot: dependabot/feedback#86

Thanks!

Hi,
I just started using your amazing app today, and the auto-merge function is working fine
but the update branch part seems not to be working.
Here is my PR : tanvn/fp-in-scala-with-ci#26
I added a wip label to the PR as wip is on my whitelist for update in my config file.
https://github.com/tanvn/fp-in-scala-with-ci/blob/master/.bulldozer.yml

Thanks in advance.

right now, a user can't see if bulldozer is enabled on a repo.

if we post a status that:

• says pending if bulldozer is waiting for something before it can try to merge (like a whitelist tag)
• says ready if bulldozer is ready to try to merge

I've deployed bulldozer via the 1.4.0 tagged docker image and setup a GithubApp for our organization. I'm now trying out the config in a toy repository. In this PR the first comment is matching the configured whitelisted "comment" and the second comment the "comment_substring". In the debug output however I see

pymor_bulldozer_1 | 2019-03-07T08:03:17.357768662Z |DEBUG| pymor/ci_playground#4 is deemed not mergeable because of unfulfilled status checks: [pymor.ci_playground]

Which seems wrong since that is the name of the only required status check and that passed.

pull.GithubContext is a bit weird about how it uses github.PullRequest objects:

1. It takes one in the constructor and several functions assume it is non-nil, but the constructor also takes 3 parameters that duplicate information from the PR
2. Body() just returns the body value of the saved PR object, but takes a context and returns an error
3. Title() just returns the title value of the saved PR object, but takes a context and returns an error

Assuming all callers have a full (or full-enough) github.PullRequest, we should only require that as an argument and then simplify any methods that only use data from the PR and make no additional API calls.

Two asks here:

• Looking for a default behavior config option (much like strategy and deleteAfterMerge) that defaults to false and if true rebases the pr when out of date if ready

• Also would be nice to have a config option that behaved more like "updateWhenOutOfDate" which didn't wait for the pr to be ready and just continually updated it

I really think that config options, as apposed to labels, should be prioritized, as it is better from an automation perspective

When a PR containing multiple commits is merged (using squash+merge), bulldozer uses the PR title as the commit message on develop.

When a PR containing a single commit is merged (using squash+merge), the commit message of that single commit appears on develop, not the PR title. This is inconsistent.

It's particularly annoying because we enforce a convention of prefixing PRs with [fix] [improvement] etc in some of our OSS repos, which then get discarded.

Example: palantir/conjure-java@8bf6a57

Somewhat related to #22

I often update PR titles to be more accurate. When I do manual merges, I use that title in the merge, but when bulldozer squashes a 1-commit PR, it ends up using the original commit message of that commit in the squashed commit message.

In the body of PR one can write resolves #xyz. Then normally, when manually closing a PR with merging, github closes thexyz issue. It would be super useful, if bulldozer do the same.

Currently, the string "Automatically merged via Bulldozer!" is hardcoded in https://github.com/palantir/bulldozer/blob/master/github/github.go#L351.

There would be more room for amusing messages if this were configurable.

Hello,

first of all I wanna thank you for this great app :)

I have one issue:
It would be helpful to have MaxPullRequestPollCount in merge.go configurable via yml file.

Reason:
we have a lot of pull requests and often the use case, that branches will be merged when developers are not in office anymore.
It would be bad, if PR already max count when no one is in office. Then, benefit of Bulldozer would be gone ;)

Additionally, there is none more thin which I do not understand:

You can see in screenshot, bulldozer merged automatically develop branch into feature branch 4 times in a row.
After this, a merge must be dne manually (385385c), because bulldozer stopped due to max count.
But after this, bulldozer tried 7 times and it worked (bulldozer also did merge into develop).
Why was max count not respected here?

Thank you in advance!

Best regards,
Stefan

k@localhost:/root/image$ buildozer android debug
# Check configuration tokens
# Ensure build layout
# Check configuration tokens
# Read available permissions from api-versions.xml
# Preparing build
# Check requirements for android
# Run 'dpkg --version'
# Cwd None
Debian 'dpkg' package management program version 1.19.0.5 (arm64).
This is free software; see the GNU General Public License version 2 or
later for copying conditions. There is NO warranty.
# Search for Git (git)
#  -> found at /usr/bin/git
# Search for Cython (cython)
#  -> found at /usr/local/bin/cython
# Search for Java compiler (javac)
#  -> found at /usr/lib/jvm/java-8-openjdk-arm64/bin/javac
# Search for Java keytool (keytool)
#  -> found at /usr/lib/jvm/java-8-openjdk-arm64/jre/bin/keytool
# Install platform
# Run "/usr/bin/python -m pip install -q --user 'appdirs' 'colorama>=0.3.3' 'jinja2' 'six'"
# Cwd None
# Apache ANT found at /home/k/.buildozer/android/platform/apache-ant-1.9.4
# Android SDK found at /home/k/.buildozer/android/platform/android-sdk-20
# Android NDK found at /home/k/.buildozer/android/platform/android-ndk-r9c
# Run '/home/k/.buildozer/android/platform/android-sdk-20/tools/android list sdk -u -e'
# Cwd /home/k/.buildozer/android/platform
SWT folder '/home/k/.buildozer/android/platform/android-sdk-20/tools/lib/aarch64' does not exist.
Please export ANDROID_SWT to point to the folder containing swt.jar for your platform.
# Command failed: /home/k/.buildozer/android/platform/android-sdk-20/tools/android list sdk -u -e
#
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2

I tried set ANDROID_SWT to x86/swt.jar
I got same error

/home/k/.buildozer/android/platform/android-sdk-20/tools/lib/x86/swt.jar does not exist

Hey all,

thank you so much for making bulldozer. We recently were looking for options to our problems, and were surprised that there is no hosted option for something like bulldozer. Would you be OK with someone hosting bulldozer as a service?

We have a repo which works with forks.
However if I put a "Update Me" label on a fork, it does not get updated.
I understand this is an issue since the branch doesn't exist on our repo, only the PR does.
Is there anyway around this?

Pre-1.0, in pr_body mode, ==MERGE_WHEN_READY== was only respected in the PR description, not in comments. For some use cases, this is actually preferred over the change introduced for #31.

For one, we have a bot that comments on PRs that have passed validation that the user can add ==MERGE_WHEN_READY== to the description for it to auto-merge. In 1.0, this comment itself would cause the PR to auto-merge, so we will have to escape that text somehow.

Secondly, updating the PR description is limited to the PR author and those with write access to the repo. We limit write access generally and users submit PRs from forks, so having it be allowed in comments, which are not as locked down, opens up this configuration more broadly than we would like.

Is it possible to configure if comments should be included or not?

I want to enable Bulldozer using a Github OAuth Token rather than the web interface, so it would be great to have a way to request a JWT over the rest api. On a similar project this was implemented as POST /api/auth/direct

Request:

{
    "access_token": "abcdefg"
}

Response:

{
    "token": "1234567"
}

In our development model we are moving to using to shared branches develop and master. We would like to use different merge options for these targets. In our case, squash for merges into develop and merge for merges into master.

Something like this would be great.

method: rebase
branch_merge_method: 
   develop: squash
   master: merge

Where the method attribute is used if there is no branch specific method that matches the current PR, otherwise the branch specific method is used.

This should be pretty simple to implement, so I'm happy to create a PR if you think this makes sense.

It would be nice if there was a way to wait for all (or possibly some) not required checks to pass before merging. Perhaps a config option like require-all-status-checks, that defaults to false for backcompat.

The use case is that there are times where a human doesn't want to be forced to wait for a status
check to pass, but does want bulldozer to wait

Hi Team,
This is the exact app which i looked for auto merge which should happen after all required branch protection settings are getting passed. I just did the setup by following the 'deployment' steps. I started the server and it listened on port 8080. I tried to have setup a branch protection settings for my 'master' branch in Github repo. Once I raised a PR request against that 'master' branch, it is quickly getting auto merged even before all 'branch protection settings/status updates happen'.

This is the config i am using against my Github Repo.

version: 1

merge:
  method: squash

  options:
    squash:
      body: "summarize_commits"

  required_statuses:
    - "default"

  delete_after_merge: true 

My questions are

1. Am i missing something?
2. Where can i find the logs? [I can see the server log with debug option where i cannot find a message other than server started and listening]

It would be great if anybody can help me out as i am struck it here.

By default, when you merge a PR through the github UI, it helpfully copies the title into the commit message, that looks something like:

Merge pull request #XYZ from <branch>

<PR title>

Bulldozer actually drops the PR title and seems to push an empty commit body:

Merge pull request #XYZ from <branch>

Proposal

It would make it easier to browse history using git-log if we could include a bit more information, e.g.:

#XYZ <PR Title>

<PR Description>

cc @alicederyn

I'm not sure what we're doing wrong here. The branches are being merged but the delete_after_merge flag is being ignored and the branches aren't being deleted. I also can't see anything in the logs that mention deletion of the branch.

Here's the project .bulldozer.yml

version: 1

merge:
  whitelist:
    branches: ["develop", "master"]
  blacklist:
    labels: ["no merge"]
    comment_substrings: ["==DO_NOT_MERGE=="]
  method: squash
  branch_method:
    develop: squash
    master: merge
  options:
    squash:
      title: "pull_request_title"
      body: "pull_request_body"
      message_delimiter: ==COMMIT_MSG==
  delete_after_merge: true

update:
  whitelist:
    branches: ["develop", "master"]
  blacklist:
    labels: ["no update"]

there are times when bulldozer encounters an error that is user-resolvable, specifically when a merge fails due to permission settings.

we currently just log these, example:

Merge rejected due to unsatisfied condition \"You're not authorized to push to this branch. Visit https://help.github.com/enterprise/2.15/user/articles/about-protected-branches/ for more information.\

in this particular case, we should be able to add a comment to the PR with feedback on what happened and possible resolution steps

Currently if any reviewer has not approved the pr, bulldozer will not merge the pr.

There is a preference specifically for this behavior (Require pull request reviews before merging) and when it isn't enabled, bulldozer shouldn't enforce this

Is it possible to add an option to rebase the current branch before merging it to develop/master?