Bitrise Webhooks processor.
Transforms various webhooks (GitHub, Bitbucket, Slack, ...) to bitrise.io's Build Trigger API format, and calls it to start a build.
Feel free to add your own webhook transform provider to this project! For more information check the How to add support for a new Provider section.
If the (commit) message includes [skip ci]
or [ci skip]
no build will be triggered.
- GitHub
- handled on the path:
/h/github/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Bitbucket webhooks V2 ("Webhooks" on the Bitbucket web UI)
- handled on the path:
/h/bitbucket-v2/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Bitbucket Server
- handled on the path:
/h/bitbucket-server/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Slack (both outgoing webhooks & slash commands)
- handled on the path:
/h/slack/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Visual Studio Team Services
- handled on the path:
/h/visualstudio/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- GitLab
- handled on the path:
/h/gitlab/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Gogs
- handled on the path:
/h/gogs/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Deveo
- handled on the path:
/h/deveo/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
- Assembla
- handled on the path:
/h/assembla/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
Service independent:
- Passthrough - reads the request headers and body and passes it to the triggered build as environment variables.
- handled on the path:
/h/passthrough/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- handled on the path:
All you have to do is register your bitrise-webhooks
URL for
a GitHub repository.
- Open your repository on GitHub.com
- Go to
Settings
of the repository - Select
Webhooks
- Click on
Add webhook
- Specify the
bitrise-webhooks
URL (.../h/github/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in thePayload URL
field - Select the events you want to trigger a webhook for
- Right now
bitrise-webhooks
supports thePush
andPull Request
events, every other webhook (triggered by another event) will be ignored.
- Click
Add webhook
That's all! The next time you push code, push a new tag or create/update a pull request a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
All you have to do is register your bitrise-webhooks
URL for
a Bitbucket repository.
- Open your repository on Bitbucket.org
- Go to
Settings
of the repository - Select
Webhooks
- Click on
Add webhook
- Specify the
bitrise-webhooks
URL (.../h/bitbucket-v2/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in theURL
field - In the Triggers section select
Choose from a full list of triggers
and the following properties:
- Repository > Push
- Pull Request > Created
- Pull Request > Updated
- Click
Save
That's all! The next time you push code, push a new tag or create/update a pull request a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
All you have to do is register your bitrise-webhooks
URL for
a Bitbucket Server repository.
- Open your repository on your self hosted Bitbucket Server instance
- Go to
Settings
of the repository - Select
Webhooks
- Click on
Create webhook
- Specify the
bitrise-webhooks
URL (.../h/bitbucket-server/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in theURL
field - In the Events section select the following properties:
- Repository > Push
- Pull Request > Opened
- Click
Save
That's all! The next time you push code, push a new tag or create a pull request a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise). Please note that Bitbucket Server doesn't send any notifications when new commits are pushed to an existing PR, so no builds will be triggered by these events.
All you have to do is register your bitrise-webhooks
URL for
a GitLab project.
- Open your project on GitLab.com
- Go to
Settings
of the project - Select
Web Hooks
- Specify the
bitrise-webhooks
URL (.../h/gitlab/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in theURL
field - In the Trigger section select:
Push events
Tag push events
Merge Request events
- Click
Add Web Hook
That's all! The next time you push code, push a new tag or create/update a merge request a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
All you have to do is register your bitrise-webhooks
URL as a Webhook in your Gogs repository.
- Open your project on your repository's hosting URL.
- Go to
Settings
of the project - Select
Webhooks
,Add Webhook
, thenGogs
. - Specify the
bitrise-webhooks
URL (.../h/gogs/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in thePayload URL
field. - Set the
Content Type
toapplication/json
. - A Secret is not required at this time.
- Set the trigger to be fired on
Just the push event
- Save the Webhook.
That's all! The next time you push code a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
All you have to do is register your bitrise-webhooks
URL for
a visualstudio.com project as a Service Hooks
integration.
You can find an official guide on visualstudio.com 's documentations site.
A short step-by-step guide:
- Open your project on visualstudio.com
- Go to the Admin/Control panel of the project
- Select
Service Hooks
- Create a service integration
- In the Service list select the
Web Hooks
option - Select the
Code pushed
event as the Trigger - In the
Filters
section select theRepository
you want to integrate - You can leave the other filters on default
- Click
Next
- On the
Action
setup form specify thebitrise-webhooks
URL (.../h/visualstudio/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in theURL
field - You can leave every other option on default
- Click
Finish
That's all! The next time you push code or push a new tag a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
All you have to do is register your bitrise-webhooks
URL for
a Deveo repository.
- Open your repository on app.deveo.com
- Go to
Hooks
of the project - Add new hook to the repository
- Select the hook type as
webhook
- Specify the
bitrise-webhooks
URL (.../h/deveo/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in theUrl
field - Type
json
in theContent type
field - Click
Save hook
That's all! The next time you push code or push a new tag a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
Follow these steps to add your bitrise-webhooks
URL to your Assembla space.
- Open your space on assembla.com or your organisation's assembla domain
- Go to the
Webhooks
section of the space - Select
Create New Webhook
- Set
Title
toBitRise Webhook
- Specify the
bitrise-webhooks
URL (.../h/assembla/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) in theExternal url
field - Select
application/json
in theContent type
field - Paste the following code to
Content
:
{"assembla": {"space": "%{space}", "action": "%{action}", "object": "%{object}"}, "message": {"title": "%{title}", "body": "%{body}", "author": "%{author}"}, "git": {"repository_suffix": "%{repository_suffix}", "repository_url": "%{repository_url}", "branch": "%{branch}", "commit_id": "%{commit_id}"}}
- Select
Code commits
and/orGit Push
in thePost updates about:
section - Click
Add
That's all! The next time you push code a build will be triggered (if you have Trigger mapping defined for the event(s) on Bitrise).
You can register the bitrise-webhooks
URL (.../h/slack/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
) as either
an Outgoing Webhook or
as a slash command for your Slack team.
Once the URL is registered check the usage section below for all the accepted and required parameters you can define in the message, and for a couple of examples.
Your message have to be in the format: key:value|key:value|...
,
where the supported keys
are:
At least one of these two parameters are required:
b
orbranch
- example:branch: master
w
orworkflow
- example:workflow: primary
Other, optional parameters:
t
ortag
- example:branch: master|tag: v1.0
c
orcommit
- example:workflow: primary|commit: eee55509f16e7715bdb43308bb55e8736da4e21e
m
ormessage
- example:branch: master|message: ship it!!
NOTE: at least either branch
or workflow
have to be specified, and of course
you can specify both if you want to. You're free to specify any number of optional parameters.
You can also send environment variables that will be available in your workflow with the format: env[KEY1]:value1|ENV[KEY2]:value2
An example with all parameters included: workflow: primary|b: master|tag: v1.0|commit:eee55509f16e7715bdb43308bb55e8736da4e21e|m: start my build!|ENV[DEVICE_NAME]:iPhone 6S|ENV[DEVICE_UDID]:82667b4079914d4aabed9c216620da5dedab630a
Simply register or use the .../h/passthrough/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
url.
Every request received on the passthrough
endpoint will trigger a build, no filtering is done or supported!.
The only limit is that neither the Headers nor the Body can be larger than 10kb.
The headers will be passed to the build in JSON serialized form, as the value of BITRISE_WEBHOOK_PASSTHROUGH_HEADERS
.
Note: headers are key value maps where the value is an array or strings, not just a single string value!
Example:
{
"Content-Type": [
"application/json"
],
"Some-Custom-Header-List": [
"first-value",
"second-value"
]
}
The body will be passed to the build as-it-is (in string/text form), as the value of BITRISE_WEBHOOK_PASSTHROUGH_BODY
.
Demo: run the server locally (e.g. with bitrise run start
) and call the .../h/passthrough/...
endpoint with curl
:
curl -X POST --data 'just a text body' -H 'Example-Header: example header value' 'http://localhost:4000/h/passthrough/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN'
by default the server will print what and where it would send (debug mode), so you should see this in the server's log:
2017/09/10 16:30:18 ===> Triggering Build: (url:https://app.bitrise.io/app/BITRISE-APP-SLUG/build/start.json)
2017/09/10 16:30:18 ====> JSON body: {"build_params":{"branch":"master","environments":[{"mapped_to":"BITRISE_WEBHOOK_PASSTHROUGH_HEADERS","value":"{\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip\"],\"Content-Length\":[\"16\"],\"Content-Type\":[\"application/x-www-form-urlencoded\"],\"Example-Header\":[\"example header value\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-For\":[\"::1\"]}","is_expand":false},{"mapped_to":"BITRISE_WEBHOOK_PASSTHROUGH_BODY","value":"just a text body","is_expand":false}]},"triggered_by":"webhook"}
- Install Go, and set up your Workspace and your $GOPATH
- Go
1.6
or newer required!
- Go
- If you want to change things, Fork this repository
git clone
the project into your GOPATH:git clone your-fork.url $GOPATH/src/github.com/bitrise-io/bitrise-webhooks
cd $GOPATH/src/github.com/bitrise-io/bitrise-webhooks
Start the server:
- Compile the
Go
codego install
- Run it with:
bitrise-webhooks -port=4000
Alternatively, with bitrise CLI:
bitrise run start
- This will start the server with gin, which does automatic re-compilation when the code changes, so you don't have to compile & restart the server manually after every code change.
By default the server will be started in Development Mode. This means that it won't send requests, it'll only print the request in the logs.
You can pass the -send-request-to
flag, or set the SEND_REQUEST_TO
environment
variable to enable sending the requests to the specified URL (every request
will be posted to the exact URL you specify).
You can switch the server into Production mode by defining the
environment variable: RACK_ENV=production
. In production mode
the server will send requests to bitrise.io,
unless you specify a send-request-to parameter.
- Register a webhook at any supported provided, pointing to your
bitrise-webhooks
server- The format should be:
http(s)://YOUR-BITRISE-WEBHOOKS.DOMAIN/h/SERVICE/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- Keep in mind that most of the providers only support SSL (HTTPS) URLs by default. If you want to use an HTTP URL you might have to set additional parameters when you register your webhook.
- The format should be:
You can use http://requestb.in to debug/check a service's webhook format.
If you have a Heroku account you can quickly create & start your own RequestBin server for free, just follow the guide on RequestBin's GitHub page.
Create a RequestBin, and register the provided URL as the Webhook URL on the service you want to test. Once the service triggers a webhook you'll see the webhook data on RequestBin.
You can pass the -send-request-to
option (or SEND_REQUEST_TO
env var) to this server to
send all the request to the specified URL (e.g. to RequestBin), instead of sending
it to bitrise.io.
git clone
the code - either the official code or your own forkcd
into the source code directoryheroku create
- Optionally, once it's created:
- To debug: set a specific URL, where every request will be sent by the server:
heroku config:set SEND_REQUEST_TO=http://request-bin-or-similar-service.com/abc123
- To send requests to bitrise.io:
- Switch the server into Production Mode:
heroku config:set RACK_ENV=production
- Make sure the
SEND_REQUEST_TO
config is no longer set. WhileSEND_REQUEST_TO
is set every request will be sent to the specified URL, it doesn't matter whether you're in Development or Production mode. You can simply set an empty value forSEND_REQUEST_TO
to disable it:heroku config:set SEND_REQUEST_TO=""
- Switch the server into Production Mode:
- To debug: set a specific URL, where every request will be sent by the server:
git push heroku master
heroku ps:scale web=1
Alternatively you can use this Heroku deploy button, if you prefer the web UI
over the heroku CLI:
Done. Your Bitrise Webhooks server is now running on Heroku.
You can open it with heroku open
- opening the root URL of the server
should present a JSON data, including the server's version
,
the current time
, the server's environment_mode
and a welcome message
.
Implement your webhook provider support code into ./service/hook/theprovider
.
Unit tests are required if you want your code to be merged into the
main bitrise-wekhooks
repository!
Once the implementation is ready add it to the selectProvider
function,
to the supportedProviders
list, in the service/hook/endpoint.go
file.
For an example you should check the service/hook/github
(single webhook triggers
only one build)
or the service/hook/bitbucketv2
(a single webhook might trigger multiple builds)
provider implementation.
- You should check
service/hook/github
for an example if a single webhook can be transformed into a single build trigger, orservice/hook/bitbucketv2
if a single webhook might trigger multiple builds - You can use http://requestb.in to debug the webhook format
of the service.
- To format & browse JSON responses you can use http://www.jsoneditoronline.org/ or a similar tool - it helps a lot in debugging & cleaning up JSON webhooks.
- Create a folder in
service/hook
, following the naming pattern of existing providers (and Go package naming conventions)- Use only lowercase ASCII letters & numbers, without any whitespace, dash, underscore, ... characters
- Create a test file (
..._test.go
) - Note: you should create a testing function for every function you add, before you'd write any code for the function!
- Split the logic into functions; usually this split should be fine (at least for starting):
- Validate the required headers (content type, event ID if supported, etc.)
- Declare your data model(s) for the Webhook data
- Create a test for the
TransformRequest
method, with checks for the required inputs (headers, event type, etc.).- You can test it with a sample webhook request string right away, but it's probably easier to write the transform utility function(s) first
- Create your transform utility function(s):
- These function(s) should get your declared Webhook data models as it's input (not the full raw request body) to make it easier to test/validate
- You can check the
transformCodePushEvent
function in theservice/hook/github
service as for an example
- Write tests for these transform utility function(s)
- It's usually a good idea to just write a list of error tests first, directly in the
_test.go
file, without an actual test implementation. - Then start to write the test implementations, one by one; write the test first, then make the code pass, then go to the next test implementation
- It's usually a good idea to just write a list of error tests first, directly in the
- Once your transform functions are well tested you should get back to the
TransformRequest
function, test & implement that too- You should include a sample webhook data & test, as you can see it in the
github
andbitbucketv2
services.
- You should include a sample webhook data & test, as you can see it in the
- Run the tests & linters with:
bitrise run test
- To run only the Go tests:
go test ./...
- To run only the tests of your own package (
github
in this example):go test ./service/hook/github/...
- To run only the Go tests:
- Once the implementation is ready you can register a path/route for the service/provider:
- Open
service/hook/endpoint.go
- Add your provider to the
supportedProviders
map- the key will be the URL (PROVIDER-ID component in the URL) this provider is registered for; URL format will be:
/h/PROVIDER-ID/BITRISE-APP-SLUG/BITRISE-APP-API-TOKEN
- the value is an object of your provider
- the key will be the URL (PROVIDER-ID component in the URL) this provider is registered for; URL format will be:
- Open
- At this point you can start the server and your provider should handle the calls as expected
- You can run the
bitrise-webhooks
executable on your server
- You can run the
- You should also send a Pull Request, so your provider will be available for others
Done! You can now test your provider on a server (check the Deployment section), and you can create a Pull Request, to have it merged with the official bitrise.io webhook processor.
Once you have a working Provider you can optionally define response transformers too. With this you can define the exact response JSON data, and the HTTP status code, both for success and for error responses.
If you don't define a response Transformer the default response provider
will be used (service/hook/default_reponse_provider.go
).
To define your own Response Provider/Transform functions you just have to
implement the functions of ResponseTransformer
(service/hook/common/common.go
).
If your Provider implements these functions it'll be used for generating the response. You have to implement every function defined in the interface, or your Provider won't be considered as an implementation of the interface and the default response provider will be used instead.
Response is always in JSON format.
If provider declares the response transformers it'll be used, and the provider is responsible for generating the response JSON.
If it doesn't provide the response transformer functions then the default response provider will be used.
The default response provider generates the following responses:
- If an error prevents any build calls then a single
{"error": "..."}
response will be generated (with HTTP code400
). - If a single success message is generated (e.g. if the hook is skipped and it's
declared as a success, instead of an error) then a
{"message": "..."}
response will be generated (with HTTP status code200
). - If at least one Bitrise Trigger call was initiated:
- All the received responses will be included as a
"success_responses": []
and"failed_responses": []
JSON arrays - And all the errors (where the response was not available / call timed out, etc.)
as a
"errors": []
JSON array (if any) - If at least one call fails:
- In case the error is not an internal error, nor an authorization or any other system error,
just the build could not be started (e.g. Trigger Map doesn't have any mapping for the trigger),
then the response code with be
200
. This is to prevent services to "disable" the Webhook, as most of the git hosting services auto-disable webhooks if too many / too frequent non2xx
responses received. - On the other hand, if it's a system error (e.g. an internal error or an authentication error)
then of course the webhook server will not return a
2xx
(success) code, but instead it'll return a400
error with as much details in the response about the error as the server can determine.
- In case the error is not an internal error, nor an authorization or any other system error,
just the build could not be started (e.g. Trigger Map doesn't have any mapping for the trigger),
then the response code with be
- If all trigger calls succeed the status code will be
201
- All the received responses will be included as a
- If the provider declares that it does not want to wait for the Trigger API response,
then a response will be returned immediately after triggering a build (calling the Trigger API),
and in the response there will be no information about the Trigger API call's response.
In this case the response is
{"did_not_wait_for_trigger_response": true}
with a HTTP200
code.- An example is the GitLab hook processor/provider, where GitLab does not store the returned response, there's no history for webhook calls, and it does retry the webhook call if the response is too slow. So in case of GitLab we don't wait for the response of the Trigger API, we just return the did not wait response.
- Re-try handling
- Docker image: auto-create & publish a Docker Image for the webhooks server, to make it easy to run it on your own server
- Bitbucket V1 (aka "Services" on the Bitbucket web UI) - not sure whether we should support this, it's already kind of deprecated, and we already support the newer, V2 webhooks.
- The Bitrise Team
- Chad Robinson -
Gogs
support - Rafael Nobre - Environment variables support in
Slack
commands - Tuomas Peippo- Skip CI feature
- Erik Poort -
Assembla
support - ChrisTitos -
Bitbucket Server
support