facebookincubator / go2chef Goto Github PK
View Code? Open in Web Editor NEWA Golang tool to bootstrap a system from zero so that it's able to run Chef to be managed
License: Apache License 2.0
A Golang tool to bootstrap a system from zero so that it's able to run Chef to be managed
License: Apache License 2.0
There is no way to provide sslclientcert
/sslclientkey
inside go2chef.step.install.linux.dnf
- hence you can't authenticate with a yum endpoint while downloading a package.
GO2CHEF 2022/06/08 19:40:32 EVENT: STEP_3_FAILURE go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli - Get "https://<redacted_uri>/yum/centos/8.x/site-packages/x86_64/Packages/chef-13.10.0-1.el7.x86_64.rpm": remote error: tls: certificate required
It would be nice to have an easy Makefile entry for M1 and newer Macs.
I'm happy to do this and test it. Should we call it darwin-apple-silicon
or darwin-arm
or darwin-arm64
?
_build/src/github.com/facebookincubator/go2chef/multilogger.go:26:3: constant 9223372036854775807 overflows int
See https://bugzilla.redhat.com/show_bug.cgi?id=1912666 for details. This came up while working on packaging go2chef for Fedora in https://bugzilla.redhat.com/show_bug.cgi?id=1912668
AWS-specific sources will often be used from an aws instance with access to the aws metadata service. This is already used by the sdk to provide credentials if they aren't in the config. However the sdk doesn't provide a way to automatically derive the current region before creating a session even though the metadata service will tell you where you are.
Instead, when using a source like s3 or secretsmanager, if region isn't specified, manually query the metadata to find our current region and use it if available. To do this, basically:
curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone
Then strip off the last character and that will be your region.
Package names are currently hardcoded, which requires patching go2chef to use it with a free/libre Chef distribution like Cinc.
Allowing passing arbitrary filenames would make this tool more broadly useful if someone wants to use it as go2puppet or such.
Because Go2Chef does not allow downgrading, it alone does not help in a scenario where it might install a stable version of Chef and you'd want it to recover if a later version of Chef caused an issue.
For instance, Chef 15.6 works with our current code and Chef 16.2 doesn't. We can't use Go2Chef to get back to Chef 15.6 without first uninstalling Chef 16.2. There should be some kind of option, such as "allow_downgrade", in the config that enables this kind of functionality.
Because of how commands are executed you can't do output redirection but you may want to. Request is to provide options to the command step to allow sending stdout and/or stderr to /dev/null (or I guess a file if someone wants?)
Git repositories are a common small-scale mechanism for distributing configs, etc. Add a Source plugin to support this in go2chef
.
Add caching support for the HTTP source. This allows us to:
Trying to build this on Debian Stable, I get:
$ make
go test ./...
# github.com/facebookincubator/go2chef/plugin/step/install/darwin/pkg
plugin/step/install/darwin/pkg/pkg.go:127:61: exit.ExitCode undefined (type *exec.ExitError has no field or method ExitCode)
note: module requires Go 1.12# github.com/facebookincubator/go2chef/plugin/step/install/linux/apt
plugin/step/install/linux/apt/apt.go:111:75: err.(*exec.ExitError).ExitCode undefined (type *exec.ExitError has no field or method ExitCode)
note: module requires Go 1.12ok github.com/facebookincubator/go2chef (cached)
ok github.com/facebookincubator/go2chef/plugin/config/embed (cached)
ok github.com/facebookincubator/go2chef/plugin/config/http (cached)
ok github.com/facebookincubator/go2chef/plugin/config/local 0.002s
ok github.com/facebookincubator/go2chef/plugin/lib/certs (cached)
ok github.com/facebookincubator/go2chef/plugin/source/http 0.039s
ok github.com/facebookincubator/go2chef/plugin/step/sanitycheck (cached)
ok github.com/facebookincubator/go2chef/plugin/step/winsanitycheck (cached)
ok github.com/facebookincubator/go2chef/util/plugconf (cached)
make: *** [Makefile:7: test] Error 2
$ go version
go version go1.11.6 linux/amd64
Either fix the module to work with go 1.11, or update the README to clarify the dependency on go 1.12
The typical usage of installing a package in dnf is like the following
{
"steps": [
{
"type": "go2chef.step.install.linux.dnf",
"name": "install chef",
"version": "15.2.20-1.el7.x86_64",
"source": {
"type": "go2chef.source.http",
"url": "https://packages.chef.io/files/stable/chef/15.2.20/el/8/chef-15.2.20-1.el7.x86_64.rpm"
}
}
]
}
This is awesome, because it doesn't depend on any repo config on the const and make the bootstrap truly independent.
This works great for packages here the version doesn't change frequently - so the config is reasonably static.
However it might not be ideal in cases where the versions are changing quickly because of CI/CD. In that case it might be desirable to specify the path of the repo instead. Something like this:
{
"steps": [
{
"type": "go2chef.step.install.linux.dnf",
"name": "install chef-solo-tools",
"version": "15.2.20-1.el7.x86_64",
"source": {
"type": "go2chef.source.repo",
"baseurl": "https://packages.chef.io/centos/8.x/x86_64/"
}
}
]
}
Right now we hunt for the 0
exit code by simply checking if err
is non-nil. If you install an MSI that is already installed it will throw 1603
which is valid... kinda sorta.
If this should be idempotent we should be able to handle either known-"good" errors or let the user of the application specify the exit codes they want to accept.
Support checksum validation for go2chef.source.http
downloads.
If using the PackageName option to override the package to install from the default ("chef"), log messages will indicate the specified package is being installed but in reality it still tries to install a package called "chef" anyway. The option is not being respected. (tested on a host using DNF)
I am presently using go2chef.step.install.linux.dnf
to install a package and version and relying on dnf to figure out the repo. But I am seeing inconsistency with how go2chef is honoring the "version" value.
go2chef doesn't honor version when the package is not installed at all.
go2chef does honor version when the package is already installed
go2chef doesn't downgrade the package if the version is installed
Repro:
########## Go2chef config ##########
[centos@ip-172-16-1-244 ~]$ cat /etc/go2chef.config-broken.json
{
"steps": [
{
"type": "go2chef.step.install.linux.dnf",
"name": "install chef-13",
"version": "13.10.0-1.el7",
"package_name": "chef"
}
]
}
########## Remove installed chef ##########
[centos@ip-172-16-1-244 ~]$ sudo dnf remove chef -y
Dependencies resolved.
================================================================================================================================================================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
================================================================================================================================================================================================================================================================================================================================================================================================================================
Removing:
chef x86_64 14.15.6-1.el7 @fb-site-packages 108 M
Removing dependent packages:
fb-chef-solo-tools noarch 20220503-100441 @any-noarch 131 k
Transaction Summary
================================================================================================================================================================================================================================================================================================================================================================================================================================
Remove 2 Packages
Freed space: 108 M
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Erasing : fb-chef-solo-tools-20220503-100441.noarch 1/2
Erasing : chef-14.15.6-1.el7.x86_64 2/2
Running scriptlet: chef-14.15.6-1.el7.x86_64 2/2
Verifying : chef-14.15.6-1.el7.x86_64 1/2
Verifying : fb-chef-solo-tools-20220503-100441.noarch 2/2
Removed:
chef-14.15.6-1.el7.x86_64 fb-chef-solo-tools-20220503-100441.noarch
Complete!
[centos@ip-172-16-1-244 ~]$ rpm -qa | grep ^chef-
[centos@ip-172-16-1-244 ~]$
########## Run go2chef - doesn't honour version ##########
[centos@ip-172-16-1-244 ~]$
[centos@ip-172-16-1-244 ~]$ sudo /usr/local/bin/go2chef --local-config /etc/go2chef.config-broken.json
___ _ __
__ _ ___|_ )__| |_ ___ / _|
/ _` / _ \/ // _| ' \/ -_) _|
\__, \___/___\__|_||_\___|_|
|___/
GO2CHEF 2022/06/08 21:27:34 loading config from source go2chef.config_source.local
GO2CHEF 2022/06/08 21:27:34 EVENT: LOGGING_INITIALIZED in go2chef.cli -
GO2CHEF 2022/06/08 21:27:34 EVENT: STEP_0_START go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli -
Last metadata expiration check: 0:42:16 ago on Wed 08 Jun 2022 08:45:19 PM UTC.
Dependencies resolved.
================================================================================================================================================================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
================================================================================================================================================================================================================================================================================================================================================================================================================================
Installing:
chef x86_64 14.15.6-1.el7 fb-site-packages 28 M
Transaction Summary
================================================================================================================================================================================================================================================================================================================================================================================================================================
Install 1 Package
Total download size: 28 M
Installed size: 108 M
Downloading Packages:
chef-14.15.6-1.el7.x86_64.rpm 30 MB/s | 28 MB 00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 30 MB/s | 28 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : chef-14.15.6-1.el7.x86_64 1/1
Running scriptlet: chef-14.15.6-1.el7.x86_64 1/1
Thank you for installing Chef Infra Client! For help getting started visit https://learn.chef.io
Verifying : chef-14.15.6-1.el7.x86_64 1/1
Installed:
chef-14.15.6-1.el7.x86_64
Complete!
GO2CHEF 2022/06/08 21:27:43 EVENT: STEP_0_COMPLETE go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli - completed successfully in 8 second(s)
GO2CHEF 2022/06/08 21:27:43 EVENT: ALL_STEPS_COMPLETE in go2chef.cli - 1 step(s) completed successfully in 8 second(s)
2022/06/08 21:27:43 temp dirs cleanup completed
[centos@ip-172-16-1-244 ~]$ rpm -qa | grep ^chef-
chef-14.15.6-1.el7.x86_64
########## Install the desired chef version ##########
[centos@ip-172-16-1-244 ~]$ sudo dnf install chef-13.10.0-1.el7 -y
Last metadata expiration check: 0:44:26 ago on Wed 08 Jun 2022 08:45:19 PM UTC.
Dependencies resolved.
================================================================================================================================================================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
================================================================================================================================================================================================================================================================================================================================================================================================================================
Downgrading:
chef x86_64 13.10.0-1.el7 cloud-common-64 49 M
Transaction Summary
================================================================================================================================================================================================================================================================================================================================================================================================================================
Downgrade 1 Package
Total download size: 49 M
Downloading Packages:
chef-13.10.0-1.el7.x86_64.rpm 31 MB/s | 49 MB 00:01
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 31 MB/s | 49 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Downgrading : chef-13.10.0-1.el7.x86_64 1/2
Running scriptlet: chef-13.10.0-1.el7.x86_64 1/2
Thank you for installing Chef!
Cleanup : chef-14.15.6-1.el7.x86_64 2/2
Running scriptlet: chef-14.15.6-1.el7.x86_64 2/2
Verifying : chef-13.10.0-1.el7.x86_64 1/2
Verifying : chef-14.15.6-1.el7.x86_64 2/2
Downgraded:
chef-13.10.0-1.el7.x86_64
Complete!
########## Run go2chef again - this time it parses the right version and honors it! ##########
[centos@ip-172-16-1-244 ~]$ sudo /usr/local/bin/go2chef --local-config /etc/go2chef.config-broken.json
___ _ __
__ _ ___|_ )__| |_ ___ / _|
/ _` / _ \/ // _| ' \/ -_) _|
\__, \___/___\__|_||_\___|_|
|___/
GO2CHEF 2022/06/08 21:30:30 loading config from source go2chef.config_source.local
GO2CHEF 2022/06/08 21:30:30 EVENT: LOGGING_INITIALIZED in go2chef.cli -
GO2CHEF 2022/06/08 21:30:30 EVENT: STEP_0_START go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli -
GO2CHEF 2022/06/08 21:30:30 INFO: third-party-source/go/github.com/facebookincubator/go2chef/plugin/step/install/linux/dnf/dnf.go:204::Package is already installed: chef-13.10.0-1.el7.x86_64, requested ^chef-13.10.0-1.el7.*
GO2CHEF 2022/06/08 21:30:30 INFO: third-party-source/go/github.com/facebookincubator/go2chef/plugin/step/install/linux/dnf/dnf.go:132::chef specified is already installed, not reinstalling
GO2CHEF 2022/06/08 21:30:30 EVENT: STEP_0_COMPLETE go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli - completed successfully in 0 second(s)
GO2CHEF 2022/06/08 21:30:30 EVENT: ALL_STEPS_COMPLETE in go2chef.cli - 1 step(s) completed successfully in 0 second(s)
2022/06/08 21:30:30 temp dirs cleanup completed
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.