rhd-gitops-example / services Goto Github PK
View Code? Open in Web Editor NEWIntra-Git repository promotions for GitOps.
License: Apache License 2.0
Intra-Git repository promotions for GitOps.
License: Apache License 2.0
The bootstrap
branch in https://github.com/rhd-gitops-example/odo shows how odo pipelines bootstrap (init)
can be used to create a gitops repo that should be compatible with our promote
verb. Check that this is still the case. Can we script a manual test for this?
In the first instance we need to go build
and go test
on PullRequests and merges. Travis is the simplest way to get this up and working.
The automerge-example should provide working, minimal RBAC and an attached service account for both the 'standalone' and 'webhooks' cases. Update the 'Prerequisites' section of the README accordingly.
odo pipeline init
creates GitOps repos that use a modified version of the gitops repository spec, as per https://github.com/rhd-gitops-example/manifest-documentation. See for example https://github.com/bigkevmcd/gitops/.
In order to support multiple environments, a path /environments/environment-name/
is added to the top of the directory structure.
Currently we use
services promote --from [path] --to [git url] --service [name]
This means, copy [name]/config/*
to /services/[name]/base/config/*
in the [git url] repo via a PR. In the new case, the target path is /environment/[environment-name]/services/[service-name]/base/config/*
and the question is, How do we determine [environment-name] when preparing the pull request, if the [service-name] subdirectory does not exist?
--env env-name
Under this issue implement option 2 and either implement option 1 or raise a follow-on issue to do so.
Also update our example projects (gitops-exampl-dev and -staging) to conform to the new format.
Also update our sample code and documentation.
services promote --from promote-demo --to https://github.com/gitops-example-dev --service promote-demo
Note the lack of an org/owner.
This gives an error of the form
(base) Adams-MBP:gitops aroberts$ services promote --from promote-demo --to https://github.com/gitops-example-dev --service promote-demo
Error: failed to clone destination repository, error: failed to clone repository, error is: could not identify repository name: https://promotion:<my token here>@github.com/gitops-example-dev. repository URL = https://github.com/gitops-example-dev. repository URL = https://github.com/gitops-example-dev. repository URL = https://github.com/gitops-example-dev
The sample in the docs directory contains a number of yaml files which are not required. Remove these files.
As well as building a docker image, we would like to make the services
binary available for download. Ideally we'd like binaries for Linux, Mac and Windows. Can we get Travis to build and publish them back to GitHub as downloads for a release?
(i.e. push a tag to GitHub, create a release, Travis release plugin publishes artifacts.)
When I try to do a services promote
command to promote to a gitops repo that's in an org, the command fails with the following error:
2020/05/15 13:49:55 failed to create a pull-request for branch workspace-git-source-local-dir-838ea, error: Not Found. repository URL = https://github.ibm.com/kabanero-org-test/test1-gitops
It creates the branch with the expected updates but does not create the PR. I gave my token pretty much all possible access.
The sample in the docs directory has a hardcoded name for the pipelinerun meaning you cannot reapply the yaml and generate new runs. Can this not be changed to generateName rather than name in the metadata?
I'm looking to contribute to this area, so I built the code here and I've ran through the example project I've found - there's no connection just yet though.
So, we've got an exemplar project at https://github.com/mnuttall/gitops-repo-testing/tree/dev (which should be under this org I guess?) and it'd be great to have the readme updated here to work with such a repository so things are more concrete than copying the sole text file.
See #2 - it's super related and also mentions including an exemplar project here
We currently publish binaries in our release process, but we also need to publish docker images.
Work with RedHat to establish whether we can publish specifically tagged images to quay.io from our travis.yml, or whether a different mechanism is required.
I'll do this one, going with a simple Go app instead. Things take a minute with the simple Pipeline atm and 16 minutes with the Buildah one I've got in a PR.
With the example I'll propose, it's one minute for the Buildah Pipeline.
Discuss: consider extending cli to permit promoting more than one service in a single PR. (e.g. new service-a depends on new service-b.)
Builds of each integration should be published somewhere official? Might be something for @bigkevmcd and @mnuttall to discuss depending on the future of this code
This issue will affect #6, which may not be able to complete without it.
This issue relates to cases in which the config yaml for a given microservice lives with the source code. A (Tekton) pipeline that builds this source repo will,
This latter step involves raising a pull request that takes config/* from the checked out version of the source repository and 'promotes' it to /services/service-name/base/config/* in the target gitops repository - frequently referred to as 'dev'.
In the first case we can assume that service-name is the name of the source repository.
Currently, promote --from /path/to/local/repo --to ur.to.target.repo --service svc-name
will look for /services/svc-name/base/config/* in the source repository, since that is the behaviour for when promoting between repositories.
We need the CLI to differentiate between promoting between gitops repositories, and from a source repo into a gitops repo. This was discussed at length in Slack today. We may also later need to differentiate between uploading service config, and that of an application.
For a first pass under this issue, we need to decide between:
/path/to/local/repo/config/*
to url.to.target.repo/services/svc-name/base/config/*
.Most of our testing so far has been on public GitHub. Re-test against GitHub Enterprise.
When I run a promote command with --debug
I should see additional output at the point of failure of the command
(base) Megans-MacBook-Pro:services [email protected]$ ./services promote --from https://github.com/Megan-Wright/gitops-repo-testing.git --to https://github.com/Megan-Wright/staging.git --service service-a --debug --branch-name megan-test-6****@@@@@
2020/04/08 13:22:39 Cloning into 'gitops-repo-testing'...
2020/04/08 13:22:41 Cloning into 'staging'...
2020/04/08 13:22:41 failed to checkout repo: failed to checkout branch megan-test-6****@@@@@ in repo https://github.com/Megan-Wright/staging.git: exit status 128
(which is the same output you get from running the command without --debug
)
Run a promote command with --debug
which will obviously fail, such as adding invalid characters to the branch name like I have done above
We're using the 'github' go-scm driver to create PullRequests. Some additional work is required to add GitLab support.
It should now be possible that on pushing a tag starting with v
, a release is made with binaries uploaded as assets, the job currently relies on GH_TOKEN being set.
In testing the https://github.com/rhd-gitops-example/services/tree/master/automerge-example @dibbles noticed errors of the form
HEAD detached from d88df61
[do-this-first : merge-pr] nothing to commit, working tree clean
when using Tekton Pipelines 0.12. These errors are not seen with Pipeline 0.11.3. The PipelineResources in the automerge-example need to be updated and re-tested with Tekton 0.12+. The failure is believed to be due to an optional refspec
option introduced in the 0.12 timeframe. See tektoncd/pipeline@8bc424d
revision
: Git revision (branch, tag, commit SHA or ref) to clone. You can use this to control what commit or branch is used. git checkout is used to switch to the revision, and will result in a detached HEAD in most cases. Use refspec along with revision if you want to checkout a particular branch without a detached HEAD. If no revision is specified, the resource will default tomaster
.
Fetch the develop branch and switch to it (not detached)
revision: develop
refspec: refs/heads/develop:refs/heads/develop
Ensure that both the 'standalone' and 'webhooks' PipelineResources are updated and tested.
When I run a promote command which fails, if I then amend this to a promote command which should pass and try to rerun it, I expect it to work without error.
After a failed promote command, any new promote command (whether valid or not) will fail because the cache hasn't been cleared, and so I have to run rm -rf ~/.promotion/cache;
before subsequent promote commands.
Run a promote command which will fail, such as adding invalid characters to the branch name, and then immediately rerun the same promote command in a way which should work.
Let's not surface this:
(base) Adams-MBP:services aroberts$ ./services promote --from https://github.com/a-roberts/gitops-repo-testing --to https://github.com/a-roberts/staging --service service-a
2020/03/31 11:12:44 failed to clone source repo https://github.com/a-roberts/gitops-repo-testing: failed to clone repo https://promotion:(token here)@github.com/a-roberts/gitops-repo-testing: exit status 128
Instead let's provide a custom error message for this scenario - not cool when, say, I'd want to demo something, accidentally do the wrong command or miss a pre-req, and now my token's there for the world to see!
--from
between local and remote cases.Further to #75 investigate why GitHub Enterprise Pull Requests don't go into 'merged' state when a merge commit is pushed, leaving code in automerge-task.yaml of the form,
# On GitHub Enterprise - not GitHub.com - the issue will still be open.
# We don't know if this is a bug or a feature. One way to work around this is with code of the form,
# export GITHUB_HOST=github.ibm.com
# hub api -XPATCH repos/mnuttall/gitops-example-dev/pulls/4 -f state=closed
# See https://github.com/github/hub/issues/1151
If unavoidable, replace the commented out section above with working code that parses the PullRequest URL to generate the hub api -XPATCH
command correctly.
Promotion currently yields PRs with a commit message, 'This is a commit message.'
Improve this to make best use of the information available. We should be able to make use of the --service argument, and --from when it's a URL.
With the improvements made at #65, we're still referencing contributor's repositories - worse still, the example repositories don't conform to our GitOps structure. So we end up Docker building them when surely that isn't what we want. Ideally, you'd just need
https://github.com/rhd-gitops-example/gitops-example-dev
https://github.com/rhd-gitops-example/gitops-example-staging
and a third repository, https://github.com/rhd-gitops-example/gitops-example-prod.
Under this issue, modify the example to use these pipelines successfully - building and pushing the code to Dockerhub for example).
We currently reference
While these are fine to be forked for testing, let's have something more official.
Accept URLs of the form, https://[email protected]/username/repository.git
and https://username:[email protected]/username/repository.git
. Don't prompt for username or password if they have all ready been provided.
More often than not, 'password' should be a token, and 'username' won't matter.
Our travis builds tolerate more warnings than is ideal. For example warnings of the form
pkg/git/mock/mock.go:203:3: Fatalf call has error-wrapping directive %w
seen when running go test ./...
locally are tolerated silently. Add golangci-lint
to our Travis build. See the script
section of https://gist.github.com/y0ssar1an/df2dab474520c4086926f672c52db139 for an example.
#32 looked good but it broke promotion from a local repository. We lack the tests for the main Promote() method that should have caught it.
Promote() results in the generation of a PullRequest. A test for this method might need to generate pull requests for both local and remote scenarios, check that they look right, and then close the PRs.
An alternative would be to introduce a --dry-run CLI flag so that we could verify what the PR would look like, were we to raise it, but stop short of doing so. This might make the tests a bit more reliable.
We need to embed the promote
cli in a Tekton task. A typical pipeline should be,
Build a Tekton Task and Pipeline that does this, and demonstrate it back to the team.
kubectl apply -k git-source/env
is currently used, so fix up that
Further to #72 our sample gitops repositories https://github.com/a-roberts/gitops-example-dev and https://github.com/a-roberts/gitops-example-staging need to be updated with top-level paths /environments/dev
and environments/staging
respectively.
Replace --branch-name
with two options:
--from-branch # branch in source repo to promote from
--to-branch. # branch in destination repo to promote to
If both options are specified and --to
is not set, set --to
to be equal to --from
.
The initial cache implementation fetches the default branch for a repository - this may not be --from-branch
, so that may also need fixing under this issue.
After one call to promote, it is necessary to rm -rf ~/.promotion/cache
before the next. Modify the CLI so that the cache is deleted by default. Add a --keep
flag to prevent cache deletion.
As a developer (or Task in a Pipeline, etc) I need to be able to say, promote service-name from/path/to/local/git/repo --to https://remote.git.repo
Either --from
needs to take a directory path as well as https:// urls, or we need a different CLI argument. I'd prefer the former but could accept the latter.
A sample gitops directory structure can be found in https://github.com/mnuttall/gitops-repo-testing. (This should itself be cleaned up and moved into the rhd-gitops-example org.)
Commands such as
services promote \
--from https://github.com/mnuttall/gitops-repo-testing \
--to https://github.com/mnuttall/staging \
--service services/service-a
creates a PR like https://github.com/mnuttall/staging/pull/1/files in which the copied files in under a root directory, 'service-a'.
What we want instead is for promote --service service-a
to effectively cp -R services/service-a
from the source to the target repository.
This enforces a convention that services are in a directory called 'services'. That's ok.
See also https://github.com/rhd-gitops-example/services#running - these docs need fixing.
The CopyService()
function in https://github.com/rhd-gitops-example/services/blob/master/pkg/git/copy.go , called from
services/pkg/avancement/service_manager.go
Lines 54 to 64 in 7bf45be
Errors such as those in the git layer currently fail with messages such as
exit status 1
This is due to our reporting stderr but swallowing stdout in functions such as execGit()
. Under this issue, add a --debug
flag to services
that prints out any stdout we are currently hiding, reports parsed command line arguments, etc. One of the standard Go loggers should be used for this purpose.
go build
, go test
are broken further to the move of git repo.
cd $GOPATH/src/github.com/rhd-gitops-example/services
go build ./cmd/services
cmd/services/main.go:9:2: cannot find package "github.com/bigkevmcd/services/pkg/avancement" in any of:
/usr/local/Cellar/go/1.12.6/libexec/src/github.com/bigkevmcd/services/pkg/avancement (from $GOROOT)
/Users/[email protected]/dev/goprojects/src/github.com/bigkevmcd/services/pkg/avancement (from $GOPATH)
cmd/services/main.go:10:2: cannot find package "github.com/bigkevmcd/services/pkg/git" in any of:
/usr/local/Cellar/go/1.12.6/libexec/src/github.com/bigkevmcd/services/pkg/git (from $GOROOT)
/Users/[email protected]/dev/goprojects/src/github.com/bigkevmcd/services/pkg/git (from $GOPATH)
cmd/services/main.go:11:2: cannot find package "github.com/mitchellh/go-homedir" in any of:
/usr/local/Cellar/go/1.12.6/libexec/src/github.com/mitchellh/go-homedir (from $GOROOT)
/Users/[email protected]/dev/goprojects/src/github.com/mitchellh/go-homedir (from $GOPATH)
cmd/services/main.go:12:2: cannot find package "github.com/tcnksm/go-gitconfig" in any of:
/usr/local/Cellar/go/1.12.6/libexec/src/github.com/tcnksm/go-gitconfig (from $GOROOT)
/Users/[email protected]/dev/goprojects/src/github.com/tcnksm/go-gitconfig (from $GOPATH)
cmd/services/main.go:13:2: cannot find package "github.com/urfave/cli/v2" in any of:
/usr/local/Cellar/go/1.12.6/libexec/src/github.com/urfave/cli/v2 (from $GOROOT)
/Users/[email protected]/dev/goprojects/src/github.com/urfave/cli/v2 (from $GOPATH)
When following the example using Tekton 0.11, all was going well until
(base) Adams-MBP:docs aroberts$ k apply -f servicepromotepipelinerun.yaml
error: error when retrieving current configuration of:
Resource: "tekton.dev/v1beta1, Resource=pipelineruns", GroupVersionKind: "tekton.dev/v1beta1, Kind=PipelineRun"
Name: "", Namespace: "promote"
Object: &{map["apiVersion":"tekton.dev/v1beta1" "kind":"PipelineRun" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":""] "generateName":"service-promote-pipelinerun-" "namespace":"promote"] "spec":map["params":[map["name":"commitId" "value":"v1"] map["name":"github-secret" "value":"promotesecret"] map["name":"github-config" "value":"promoteconfigmap"] map["name":"to" "value":"https://github.com/<your github org>/<your github repo>.git"] map["name":"service" "value":"service-a"]] "pipelineRef":map["name":"service-promote-pipeline"] "resources":[map["name":"git-source" "resourceRef":map["name":"git-app-repo"]] map["name":"docker-image" "resourceRef":map["name":"docker-app-image"]]] "serviceAccountName":"demo" "workspaces":[map["name":"repo-space" "persistentVolumeClaim":map["claimName":"repopvc"]]]]]}
from server for: "servicepromotepipelinerun.yaml": resource name may not be empty
This is using Tekton 0.11 as proven with
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller:v0.11.0-rc2@sha256:97c8424c6091ab3ffe5a56c4c564e0342279f8c16f3b51ddc66bf3f22e457a78
As an extra bonus, don't use the now deprecated PipelineResources, see https://github.com/tektoncd/pipeline/blob/master/docs/migrating-v1alpha1-to-v1beta1.md#pipelineresources-and-catalog-tasks which mentions the equivalent types we could use in Tasks.
Note that we may want to have a separate folder for the v1beta resources so anyone stuck on alpha for whatever reason could still use that - something to think about.
With 379c24b merged it'd be useful to have the PipelineRun/TaskRun in the example being runnable through tkn
as opposed to creating the PipelineRun from yaml.
We believe this is blocked by tektoncd/cli#940 and this is the command we have so far that'd we like to prove works and then document.
tkn pipeline start service-promote-pipeline --resource git-source=git-app-repo --resource docker-image=docker-app-image --param commitId=v1 --param github-secret=promote-secret --param github-config=promoteconfigmap --param to="https://github.com/Megan-Wright/staging.git" --param service=service-a --workspace name=repo-space,secret=promote-secret
The reproduce for this behaviour is to follow https://github.com/rhd-gitops-example/services/tree/master/tekton-example and do the first PipelineRun creation, then the second TaskRun creation.
You'll notice that for the service-promote
Task, we get a pull request mentioning a local filesystem.
I believe this is because we clone the project to the workspace Tekton uses so it's not a valid URL (it's, for example, /workspace/git-app-repo and not https://github.com/myorg/repo.git), and thus we assume the user is going down a local promotion path.
This isn't what I expected though, I thought it would be a promote from repository X to repository Y.
The problem PR: https://github.com/a-roberts/gitops-test/pull/2
And not a problem PR (from repo to repo), you get a-roberts/gitops-repo-testing#1 which is better.
Note, I've opened an issue to use exemplar repositories (both in terms of name and contents) for the Tekton example at #68.
I'm going through it now and it's particularly clunky but can easily be improved - I'd rather set a bunch of variables once, have the variables be passed through to the yaml (under a templates
folder perhaps), and be good to go - or simply move everything I need to apply to be under the one folder (instead of docs
) and everything to change goes in another.
One fancy approach would be to have a bash script, a variables file, and then do some sedding instead of having users replace xxx
in six files.
Tidy the documentation up and make it as short, clean, and, in one go, kubectl apply
able as possible, perhaps using tkn
(base) Adams-MBP:services aroberts$ ./services --debug promote --from https://github.com/a-roberts/gitops-repo-testing --to https://github.com/a-roberts/staging --service service-a
2020/03/31 11:27:59 fatal: destination path 'gitops-repo-testing' already exists and is not an empty directory.
(base) Adams-MBP:services aroberts$ ./services promote --from https://github.com/a-roberts/gitops-repo-testing --to https://github.com/a-roberts/staging --service service-a --debug
Incorrect Usage: flag provided but not defined: -debug
NAME:
services promote - promote from one environment to another
USAGE:
services promote [command options] [arguments...]
OPTIONS:
--from value source Git repository
--to value destination Git repository
--service value service name to promote
--branch-name value the name to use for the newly created branch (default: "test-branch")
--cache-dir value where to cache Git checkouts (default: "~/.promotion/cache")
--commit-name value the name to use for commits when creating branches [$COMMIT_NAME]
--commit-email value the email to use for commits when creating branches [$COMMIT_EMAIL]
--help, -h show help (default: false)
2020/03/31 11:31:28 flag provided but not defined: -debug
You can see I had to provide it at globally (so at the top level, ./services --debug) and this means I can't stick it on the end. There's been good discussion in Slack when I raised this (so @bigkevmcd and @mnuttall, lemme know if I'm misquoting you or I've missed something important!).
Kevin suggested we could move to using cobra
(I think that's an aside), and even provided the change we'd want to make (so at the time of this issue - simply from the global to promote flag list):
https://github.com/rhd-gitops-example/services/blob/master/cmd/services/main.go#L37 to https://github.com/rhd-gitops-example/services/blob/master/cmd/services/main.go#L85
so I think this makes for a useful first issue for somebody if we did want to make it so.
Review the instructions in the doc directory and simplify the instructions where/if possible - additional comments might be added to yaml files or classifications made to make it easier for a novice to know what needs changing.
We now have docker images published to quay.io/redhat-developer/gitops-cli
. Re-test the Tekton sample in docs/ with this image, and update the yaml and readme once it's confirmed to be working.
odo
runs on Windows and we're close I think to having Windows binaries/tests being done, see https://travis-ci.org/github/rhd-gitops-example/services/jobs/678499631 which indicates classic portability problems (differences in path separators and newlines)
Under this issue - using Travis as the dev/test environment if you must, add windows
into the .travis.yml as an operating system and fix up our tests. Bonus points if you do manual testing as well to check our test coverage is good and all is well 😄
Copying a comment from @mnuttall on Slack when I asked about this (so, incase it's not easy):
One IBM colleague (Tom Seelbach) noticed that we weren’t using a viper/cobra base. He mentioned that they had created their CLI project by copying odo and reducing it to a shell , and that that framework gave them multiplatform support very easily. I mention it in case Windows-compatibility becomes difficult on the current code base.
https://github.com/rhd-gitops-example/services/blob/master/tekton-example/resources/build-task.yaml mounts the docker socket. Replace this with buildah
as per https://github.com/tektoncd/catalog/tree/master/buildah
Each promotion PR needs its own branch. The CLI user should not be required to specify --branch [new branch]
on each request. If --branch is not specified, a temporary, unique branch should be created.
@bigkevmcd writes,
oc has been released with plugin support
so, if you were to rename thecmd/services
tocmd/oc-services
or symlink it or do something else you'd have a plugin
Test this out! Does oc
add any value to running services
either in a docker container on the command line?
I've had a look through the code and readme and noticed https://github.com/rhd-gitops-example/services/blob/master/pkg/avancement/service_manager.go#L64
So I assume deployment.txt is just an example of what we're using atm as a PoC, where do we think this should do instead going forward?
I'm thinking it should pickup whatever folks we'd like, perhaps specified via some args and/or a file, so discussion welcome
(nit: should the package be advancement
not avancement
?)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.