Safe-settings – an app to manage policy-as-code and apply repository settings across an organization.
-
In
safe-settings, all the settings are stored centrally in anadminrepo within the organization. Unlike the GitHub Repository Settings App, the settings files cannot be in individual repositories.It is possible specify a custom repo instead of the
adminrepo withADMIN_REPO. See Environment variables for more details. -
The settings in the default branch are applied. If the settings are changed on a non-default branch and a PR is created to merge the changes, the app runs in a
dry-runmode to evaluate and validate the changes. Checks pass or fail based on thedry-runresults. -
In
safe-settingsthe settings can have 2 types of targets:org- These settings are applied to the organization.Org-targeted settings are defined in.github/settings.yml. Currently, onlyrulesetsare supported asorg-targeted settings.repo- These settings are applied to repositories.
-
For the
repo-targeted settings, there can be 3 levels at which the settings are managed:-
Org-level settings are defined in.github/settings.ymlIt is possible to override this behavior and specify a different filename for the
settings.ymlfile withSETTINGS_FILE_PATH. Similarly, the.githubdirectory can be overridden withCONFIG_PATH. See Environment variables for more details. -
Suborglevel settings. Asuborgis an arbitrary collection of repos belonging to projects, business units, or teams. Thesuborgsettings reside in a yaml file for eachsuborgin the.github/suborgsfolder.In
safe-settings,suborgscould be groups of repos based onrepo names, orteamswhich the repos have collaborators from, orcustom property valuesset for the repos -
Repolevel settings. They reside in a repo specific yaml in.github/reposfolder
-
-
It is recommended to break the settings into
org-level,suborg-level, andrepo-level units. This will allow different teams to define and manage policies for their specific projects or business units. WithCODEOWNERS, this will allow different people to be responsible for approving changes in different projects.
Note
The suborg and repo level settings directory structure cannot be customized.
Settings files must have a .yml extension only. For now, the .yaml extension is ignored.
The App listens to the following webhook events:
-
push: If the settings are created or modified, that is, if push happens in the
defaultbranch of theadminrepo and the file added or changed is.github/settings.ymlor.github/repos/*.ymlor.github/suborgs/*.yml, then the settings would be applied either globally to all the repos, or specific repos. For each repo, the settings that are actually applied depend on the default settings for the org, overlaid with settings for the suborg that the repo belongs to, overlaid with the settings for that specific repo. -
repository.created: If a repository is created in the org, the settings for the repo - the default settings for the org, overlaid with settings for the suborg that the repo belongs to, overlaid with the settings for that specific repo - is applied.
-
branch_protection_rule: If a branch protection rule is modified or deleted,
safe-settingswillsyncthe settings to prevent any unauthorized changes. -
repository.edited: For e.g. If the default branch is renamed, or if topics change,
safe-settingswillsyncthe settings, to prevent any unauthorized changes. -
repository.renamed: If a repository is renamed, the default behavior is safe-settings will ignore this (for backward-compatibility). If
BLOCK_REPO_RENAME_BY_HUMANenv variable is set to true,safe-settingswill revert the repo to the previous name unless it is renamed using abot. If it is renamed using abot, it will try to copy the existing<old-repo>.ymlto<new-repo>.ymlso that the repo config yml stays consistent. If a <new-repo.yml> file already exists, it doesn't create a new one. -
pull_request.opened, pull_request.reopened, check_suite.requested: If the settings are changed, but it is not in the
defaultbranch, and there is an existing PR, the code will validate the settings changes by running safe-settings innopmode and update the PR with thedry-runstatus. -
repository_ruleset: If the
rulesetsettings are modified in the UI manually,safe-settingswillsyncthe settings to prevent any unauthorized changes. -
member_change_events: If a member is added or removed from a repository,
safe-settingswillsyncthe settings to prevent any unauthorized changes. -
member', team.added_to_repository, team.removed_from_repository, team.edited:
safe-settingswillsyncthe settings to prevent any unauthorized changes. -
custom_property_values: If new repository properties are set for a repository,
safe-settingswill run to so that if a sub-org config is defined by that property, it will be applied for the repo
If you rename a <repo.yml> that corresponds to a repo, safe-settings will rename the repo to the new name. This behavior will take effect whether the env variable BLOCK_REPO_RENAME_BY_HUMAN is set or not.
safe-settings can be turned on only to a subset of repos by specifying them in the runtime settings file, deployment-settings.yml. If no file is specified, then the following repositories - 'admin', '.github', 'safe-settings' are exempted by default.
A sample of deployment-settings file is found here.
To apply safe-settings only to a specific list of repos, add them to the restrictedRepos section as include array.
To ignore safe-settings for a specific list of repos, add them to the restrictedRepos section as exclude array.
Note
The include and exclude attributes support as well regular expressions.
By default they look for regex, Example include: ['SQL'] will look apply to repos with SQL and SQL_ and SQL- etc if you want only SQL repo then use include:['^SQL$']
Admins setting up safe-settings can include custom rules that would be validated before applying a setting or overriding a broader scoped setting.
The code has to return true if validation is successful, or false if it isn't.
If the validation fails, the error attribute specified would be used to create the error message in the logs or in the PR checks.
The first use case is where a custom rule has to be applied for a setting on its own. For e.g. No collaborator should be given admin permissions.
For this type of validation, admins can provide custom code as configvalidators which validates the setting by itself.
For e.g. for the case above, it would look like:
configvalidators:
- plugin: collaborators
error: |
`Admin role cannot be assigned to collaborators`
script: |
console.log(`baseConfig ${JSON.stringify(baseconfig)}`)
return baseconfig.permission != 'admin'For convenience this script has access to a variable, baseconfig, that contains the setting that is be applied.
The second use case is where custom rule has to be applied when a setting in the org or suborg level is being overridden. Such as, when default branch protection is being overridden.
For this type of validation, admins can provide custom code as overridevalidators. The script can access two variables, baseconfig and overrideconfig which represent the base setting and the setting that is overriding it.
A sample would look like:
overridevalidators:
- plugin: branches
error: |
`Branch protection required_approving_review_count cannot be overidden to a lower value`
script: |
console.log(`baseConfig ${JSON.stringify(baseconfig)}`)
console.log(`overrideConfig ${JSON.stringify(overrideconfig)}`)
if (baseconfig.protection.required_pull_request_reviews.required_approving_review_count && overrideconfig.protection.required_pull_request_reviews.required_approving_review_count ) {
return overrideconfig.protection.required_pull_request_reviews.required_approving_review_count >= baseconfig.protection.required_pull_request_reviews.required_approving_review_count
}
return trueA sample of deployment-settings file is found here.
When there are 1000s of repos to be managed -- and there is a global settings change -- safe-settings will have to work efficiently and only make the necessary API calls.
The app also has to complete the work within an hour: the lifetime of the GitHub app token.
To address these constraints the following design decisions have been implemented:
Probotautomatically handlesrateandabuselimits.- Instead of loading all the repo contents from
.github/repos/*, it will selectively load the specific repo file based on whichreposettings has changed, or a subset of the repo files associated withsuborgsettings that has changed. The only time all the repo files will be loaded is if there is aglobalsettings file change. - The PR check will only provide a summary of errors and changes. (Providing the details of changes for 1000s of repos will error out.)
- To ensure it handles updates to GitHub intelligently, it will compare the changes with the settings in GitHub, and will call the API only if there are
realchanges.
To determine if there are real changes, the code will generate a detailed list of additions, modifications, and deletions compared to the settings in GitHub:
For e.g:
If the settings is:
{
"branches": [
{
"name": "master",
"protection": {
"required_pull_request_reviews": {
"required_approving_review_count": 2,
"dismiss_stale_reviews": false,
"require_code_owner_reviews": true,
"dismissal_restrictions": {}
},
"required_status_checks": {
"strict": true,
"contexts": []
},
"enforce_admins": false
}
}
]
}and the settings in GitHub is:
{
"branches": [
{
"name": "master",
"protection": {
"url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection",
"required_status_checks": {
"url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/required_status_checks",
"strict": true,
"contexts": [],
"contexts_url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/required_status_checks/contexts",
"checks": []
},
"restrictions": {
"url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/restrictions",
"users_url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/restrictions/users",
"teams_url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/restrictions/teams",
"apps_url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/restrictions/apps",
"users": [],
"teams": [],
"apps": []
},
"required_pull_request_reviews": {
"url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/required_pull_request_reviews",
"dismiss_stale_reviews": true,
"require_code_owner_reviews": true,
"required_approving_review_count": 2,
"dismissal_restrictions": {
"url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/dismissal_restrictions",
"users_url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/dismissal_restrictions/users",
"teams_url": "https://api.github.com/repos/decyjphr-org/test/branches/develop/protection/dismissal_restrictions/teams",
"users": [],
"teams": []
}
},
"required_signatures": false,
"enforce_admins": false,
"required_linear_history": false,
"allow_force_pushes": {
"enabled": false
},
"allow_deletions": false,
"required_conversation_resolution": false
}
}
]
}the results of comparison would be:
{
"additions": {},
"modifications": {
"branches": [
{
"protection": {
"required_pull_request_reviews": {
"dismiss_stale_reviews": false
}
},
"name": "master"
}
]
},
"deletions": {},
"hasChanges": true
}The App can be configured to apply the settings on a schedule. This could be a way to address configuration drift since webhooks are not always guaranteed to be delivered.
To periodically converge the settings to the configuration, set the CRON environment variable. See Environment variables for more details.
Safe-settings explicitly looks in the admin repo in the organization for the settings files. The admin repo could be a restricted repository with branch protections and CODEOWNERS
In that set up, when changes happen to the settings files and there is a PR for merging the changes back to the default branch in the admin repo, safe-settings will run checks – which will run in nop mode and produce a report of the changes that would happen, including the API calls and the payload.
For e.g. If we have override validators that will fail if org-level branch protections are overridden at the repo or suborg level with a lesser number of required approvers, here is an screenshot of what users will see in the PR.
Note
If you don't want the PR message to have these details, they can be turned off with CREATE_PR_COMMENT. See Environment variables for more details.
Here is a screenshot of what the users will see in the checkrun page:
The app creates a Check at the end of its processing to indicate if there were any errors. The Check is called safe-settings and corresponds to the latest commit on the default branch of the admin repo.
Here is an example of a checkrun result:
And the checkrun page will look like this:
The settings file can be used to set the policies at the org, suborg or repo level.
The following can be configured:
Repository settings- home page, url, visibility, has_issues, has_projects, wikis, etc.Default branch- naming and renamingTopicsCustom propertiesTeams and permissionsCollaborators and permissionsIssue labelsMilestonesBranch protections- if the name of the branch isdefaultin the settings, it is applied to thedefaultbranch of the repo.AutolinksRepository name validationusing regex patternRulesets
It is possible to provide an include or exclude settings to restrict the collaborators, teams, labels to a list of repos or exclude a set of repos for a collaborator.
See docs/sample-settings/settings.yml for a sample settings file.
In addition to the values in the file above, the settings file can have some additional values:
force_create: This is set in the repo-level settings to force create the repo if the repo does not exist.template: This is set in the repo-level settings, and is used with theforce_createflag to use a specific repo template when creating the reposuborgrepos: This is set in the suborg-level settings to define an array of repos. This field can also take aglobpattern to allow wild-card expression to specify repos in a suborg. For e.g.test*would includetest,test1,testing, etc.- The
suborgteamssection contains a list of teams, and all the repos belonging to the teams would be part of thesuborg
You can pass environment variables; the easiest way to do it is via a .env file.
CRONyou can pass a cron input to runsafe-settingsat a regular schedule. This is based on node-cron. For eg.
# ┌────────────── second (optional)
# │ ┌──────────── minute
# │ │ ┌────────── hour
# │ │ │ ┌──────── day of month
# │ │ │ │ ┌────── month
# │ │ │ │ │ ┌──── day of week
# │ │ │ │ │ │
# │ │ │ │ │ │
# * * * * * *
CRON=* * * * * # Run every minute
- Logging level can be set using
LOG_LEVEL. For e.g.
LOG_LEVEL=trace
- Configure the source repository using
ADMIN_REPO(default isadmin). For e.g.
ADMIN_REPO=safe-settings-config
- Configure the config path using
CONFIG_PATH(default is.github). For e.g.
CONFIG_PATH=.github
- Configure the settings file path using
SETTINGS_FILE_PATH(default issettings.yml). For e.g.
SETTINGS_FILE_PATH=settings.yml
- Configure the deployment settings file path using
DEPLOYMENT_CONFIG_FILE(default isdeployment-settings.yml). For e.g.
DEPLOYMENT_CONFIG_FILE=deployment-settings.yml
- Enable the pull request comment using
ENABLE_PR_COMMENT(default istrue). For e.g.
ENABLE_PR_COMMENT=true
- Block repository renaming manually using
BLOCK_REPO_RENAME_BY_HUMAN(default isfalse). For e.g.
BLOCK_REPO_RENAME_BY_HUMAN=true
- Besides the above settings files, the application can be bootstrapped with
runtimesettings. - The
runtimesettings are configured indeployment-settings.ymlthat is in the directory from where the GitHub app is running. - Currently the only setting that is possible are
restrictedRepos: [... ]which allows you to configure a list of repos within yourorgthat are excluded from the settings. If thedeployment-settings.ymlis not present, the following repos are added by default to therestrictedrepos list:'admin', '.github', 'safe-settings'
- Label color can also start with
#, e.g.color: '#F341B2'. Make sure to wrap it with quotes! - Each top-level element under branch protection must be filled (eg:
required_pull_request_reviews,required_status_checks,enforce_adminsandrestrictions). If you don't want to use one of them you must set it tonull(see comments in the example above). Otherwise, none of the settings will be applied. - The precedence order is repository > suborg > org (.github/repos/.yml > .github/suborgs/.yml > .github/settings.yml
-
Create an
adminrepo (or an alternative of your choosing) within your organization. Remember to setCONFIG_REPOif you choose something other thanadmin. See Environment variables for more details. -
Add the settings for the
org,suborgs, andrepos. Sample files can be found here.
safe-settings is licensed under the ISC license
safe-settings uses 3rd party libraries, each with their own license. These are found here.
![dependabot-preview[bot] avatar](https://avatars.githubusercontent.com/in/2141?v=4 "dependabot-preview[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![greenkeeper[bot] avatar](https://avatars.githubusercontent.com/in/505?v=4 "greenkeeper[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent