alessiodionisi / terraform-provider-k0s Goto Github PK

Hi,

Thanks once more for this plugin.

I 've been trying out the plugin with some arm VMs and I got bitten by the following:

goroutine 385 [running]:
github.com/k0sproject/k0sctl/phase.(*PrepareArm).etcdUnsupportedArch(0xc000368280?, 0xc0006d4e00)
	github.com/k0sproject/[email protected]/phase/arm_prepare.go:48 +0xce
github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster.Hosts.ParallelEach.func1(0xc0006d4e00)
	github.com/k0sproject/[email protected]/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster/hosts.go:115 +0xc4
created by github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster.Hosts.ParallelEach
	github.com/k0sproject/[email protected]/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster/hosts.go:114 +0x24e
panic: assignment to entry in nil map

Looking a bit at the code in question and trying to reproduce using the k0sctl binary as is, I am a bit at a loss. I can't reproduce it running the k0sctl, that one works fine.

The code suggests to me that what's missing is the Environment map (thus being nil) but I am not yet sure why vanilla k0sctl calls have it and the plugin doesn't.

Hi Alessio Dionisi,

Great that you made a K0s provider in TF. I use TF a lot and work with K0s daily, but I'm lacking go skills.. ;-(

Can you document and/or tell me how to get the k0s_cluster provider working (including how to deal with go).
I have go installed but it rains errors as per my lack of knowledge....

[ec2-user@ip-172-31-24-202 cluster_resource]$ vi cluster_resource.go
[ec2-user@ip-172-31-24-202 cluster_resource]$ go build
cluster_resource.go:7:2: cannot find package "github.com/hashicorp/terraform-plugin-framework/diag" in any of:
/usr/lib/golang/src/github.com/hashicorp/terraform-plugin-framework/diag (from $GOROOT)
/home/ec2-user/go/src/github.com/hashicorp/terraform-plugin-framework/diag (from $GOPATH)
cluster_resource.go:8:2: cannot find package "github.com/hashicorp/terraform-plugin-framework/tfsdk" in any of:
/usr/lib/golang/src/github.com/hashicorp/terraform-plugin-framework/tfsdk (from $GOROOT)
/home/ec2-user/go/src/github.com/hashicorp/terraform-plugin-framework/tfsdk (from $GOPATH)
cluster_resource.go:9:2: cannot find package "github.com/hashicorp/terraform-plugin-framework/types" in any of:
/usr/lib/golang/src/github.com/hashicorp/terraform-plugin-framework/types (from $GOROOT)
/home/ec2-user/go/src/github.com/hashicorp/terraform-plugin-framework/types (from $GOPATH)
cluster_resource.go:10:2: cannot find package "github.com/hashicorp/terraform-plugin-go/tftypes" in any of:
/usr/lib/golang/src/github.com/hashicorp/terraform-plugin-go/tftypes (from $GOROOT)
/home/ec2-user/go/src/github.com/hashicorp/terraform-plugin-go/tftypes (from $GOPATH)
cluster_resource.go:11:2: cannot find package "github.com/k0sproject/dig" in any of:
/usr/lib/golang/src/github.com/k0sproject/dig (from $GOROOT)
/home/ec2-user/go/src/github.com/k0sproject/dig (from $GOPATH)
cluster_resource.go:12:2: cannot find package "github.com/k0sproject/k0sctl/phase" in any of:
/usr/lib/golang/src/github.com/k0sproject/k0sctl/phase (from $GOROOT)
/home/ec2-user/go/src/github.com/k0sproject/k0sctl/phase (from $GOPATH)
cluster_resource.go:13:2: cannot find package "github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1" in any of:
/usr/lib/golang/src/github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1 (from $GOROOT)
/home/ec2-user/go/src/github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1 (from $GOPATH)
cluster_resource.go:14:2: cannot find package "github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster" in any of:
/usr/lib/golang/src/github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster (from $GOROOT)
/home/ec2-user/go/src/github.com/k0sproject/k0sctl/pkg/apis/k0sctl.k0sproject.io/v1beta1/cluster (from $GOPATH)
cluster_resource.go:15:2: cannot find package "github.com/k0sproject/rig" in any of:
/usr/lib/golang/src/github.com/k0sproject/rig (from $GOROOT)
/home/ec2-user/go/src/github.com/k0sproject/rig (from $GOPATH)
cluster_resource.go:16:2: cannot find package "gopkg.in/yaml.v2" in any of:
/usr/lib/golang/src/gopkg.in/yaml.v2 (from $GOROOT)
/home/ec2-user/go/src/gopkg.in/yaml.v2 (from $GOPATH)

Hi Alessio,

Thanks for this great TF Provider, I was wondering if this provider could be updated to allow ssh key's to be passed directly by value.

I have a TF Pipeline that I'm trying to use native terraform features only and the key_path under ssh properties is the last thing stopping me from integrating the TF provider into that pipeline.

If you would like this feature, I can provide a pull request with it implemented (I just want to confirm that you'd want the feature added before developing it).

Thanks for creating this terraform provider!

I'm currently using k0s with a custom CNI and to install I usually use k0sctl apply --no-wait because by default k0sctl waits for all the worker nodes to be in a Ready state before exiting. This flag --no-wait overrides this behavior and lets the installer succeed after installing k0s without waiting for the worker nodes to enter a Ready state.

Is there a way to do the same with this terraform provider?

terraform init fails with the following error:

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/k0s...
╷
│ Error: Failed to query available provider packages
│ 
│ Could not retrieve the list of available versions for provider hashicorp/k0s: provider registry registry.terraform.io does not have a provider named registry.terraform.io/hashicorp/k0s
│ 
│ All modules should specify their required_providers so that external consumers will get the correct providers when using a module. To see which modules are currently depending on hashicorp/k0s, run the
│ following command:
│     terraform providers

With TF_LOG=trace the logs are more helpful

2022-09-03T22:11:27.737+0200 [INFO]  Terraform version: 1.2.8
2022-09-03T22:11:27.737+0200 [DEBUG] using github.com/hashicorp/go-tfe v1.0.0
2022-09-03T22:11:27.737+0200 [DEBUG] using github.com/hashicorp/hcl/v2 v2.12.0
2022-09-03T22:11:27.737+0200 [DEBUG] using github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2
2022-09-03T22:11:27.737+0200 [DEBUG] using github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734
2022-09-03T22:11:27.737+0200 [DEBUG] using github.com/zclconf/go-cty v1.11.0
2022-09-03T22:11:27.737+0200 [INFO]  Go runtime version: go1.18.1
2022-09-03T22:11:27.737+0200 [INFO]  CLI args: []string{"/var/lib/tfenv/versions/1.2.8/terraform", "init"}
2022-09-03T22:11:27.737+0200 [TRACE] Stdout is a terminal of width 213
2022-09-03T22:11:27.737+0200 [TRACE] Stderr is a terminal of width 213
2022-09-03T22:11:27.737+0200 [TRACE] Stdin is a terminal
2022-09-03T22:11:27.737+0200 [DEBUG] Attempting to open CLI config file: /home/user/.terraformrc
2022-09-03T22:11:27.737+0200 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2022-09-03T22:11:27.737+0200 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2022-09-03T22:11:27.737+0200 [DEBUG] ignoring non-existing provider search directory /home/user/.terraform.d/plugins
2022-09-03T22:11:27.737+0200 [DEBUG] ignoring non-existing provider search directory /home/user/.local/share/terraform/plugins
2022-09-03T22:11:27.737+0200 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2022-09-03T22:11:27.737+0200 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2022-09-03T22:11:27.737+0200 [INFO]  CLI command args: []string{"init"}

Initializing the backend...
2022-09-03T22:11:27.738+0200 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2022-09-03T22:11:27.738+0200 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2022-09-03T22:11:27.738+0200 [DEBUG] New state was assigned lineage "b3b0bc2b-fa67-f605-724c-eec09c09b821"
2022-09-03T22:11:27.738+0200 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2022-09-03T22:11:27.738+0200 [TRACE] Meta.Backend: instantiated backend of type <nil>
2022-09-03T22:11:27.739+0200 [DEBUG] checking for provisioner in "."
2022-09-03T22:11:27.739+0200 [DEBUG] checking for provisioner in "/var/lib/tfenv/versions/1.2.8"
2022-09-03T22:11:27.739+0200 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2022-09-03T22:11:27.739+0200 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2022-09-03T22:11:27.739+0200 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2022-09-03T22:11:27.739+0200 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2022-09-03T22:11:27.739+0200 [TRACE] statemgr.Filesystem: read nil snapshot

Initializing provider plugins...
- Finding latest version of hashicorp/k0s...
2022-09-03T22:11:27.739+0200 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2022-09-03T22:11:27.739+0200 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
2022-09-03T22:11:27.954+0200 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/k0s/versions
2022-09-03T22:11:27.954+0200 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/k0s/versions
2022-09-03T22:11:28.187+0200 [DEBUG] GET https://registry.terraform.io/v1/providers/-/k0s/versions
2022-09-03T22:11:28.187+0200 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/k0s/versions
╷
│ Error: Failed to query available provider packages
│ 
│ Could not retrieve the list of available versions for provider hashicorp/k0s: provider registry registry.terraform.io does not have a provider named registry.terraform.io/hashicorp/k0s
│ 
│ All modules should specify their required_providers so that external consumers will get the correct providers when using a module. To see which modules are currently depending on hashicorp/k0s, run the
│ following command:
│     terraform providers
╵

To be able to play with k0sctl outside of this provider control (e.g. to troubleshoot), it would be useful to have a property to get the full k0sctl configuration.

Maybe, call it, k0sctlconfig (akin to kubeconfig).

I think it should be the yaml serialized version of the getK0sctlConfig function return value.

Some time ago, k0sctl added support for node removal.

This provider calls the necessary phase to reset controllers, but it doesn't prepare hosts list, so they can be removed. Data structure ClusterResourceModelHost misses Reset field, and there's no logic that would translate host removal from state to flag update, so it can be picked up by phase manager.

It's quite problematic, when after removal, a new host is added with the same IP, as this is the unique ID for many k0s structures - it results in split-brain. The cluster still tries to connect to a new VM using IP that was not removed (mainly from etcd) and the new VM is stuck on cluster init phase, but serves requests immediately. Control-plane HA requires a load-balancer, so without sophisticated checks it can easily serve two clusters at the same time.

As per docs, the workaround seems to be to manually execute k0s etcd leave --peer-address IP_ADDR on an alive node - in most cases the node we want to delete, but it gets tricky if we're rebuilding a crashed VM. More so, since destroy time provisioners in TF only work with clean destroy - not even with taint.

I know that it may not be standard, but, it would be great to add support for ssh using password instead of keys.

      ssh = {
        address  = "10.0.0.1"
        port     = 22
        user     = "root"
        key_path = ""
        password = "myPassword"
      }

Similar to how it works for remote-exec:

resource "null_resource" "example" {
  provisioner "remote-exec" {
    connection {
      type        = local.use_password_auth ? "ssh" : "ssh"
      private_key = local.use_password_auth ? null : var.ssh_private_key
      password    = local.use_password_auth ? var.ssh_password : "MYPASSWORD"
      user        = var.ssh_user
      host        = var.ssh_host
      port        = var.ssh_port
    }
    inline = [
    ]
  }
}

I checked the file internal/provider/schema.go but is not present for now, so not sure how easy will be implement it:

type k0sctlSchemaModelSpecHostSSH struct {
	Address types.String `tfsdk:"address"`
	KeyPath types.String `tfsdk:"key_path"`
	User    types.String `tfsdk:"user"`
	Port    types.Int64  `tfsdk:"port"`
}

Btw, great work!

Is there any way to get the access data? (E.g. cluster_ca_certificate, client_certificate and client_key) . If it is not the case, then I am willing to submit a pull request with a data source (https://developer.hashicorp.com/terraform/plugin/framework/data-sources).

Hi,

Thanks for this provider.

I 've been trying it out and I met the following issue. Trying to use it in a provider that doesn't give out private address by default, but rather public ones, I got the following

privateAddress  is not unique: k0s-controller-0
privateAddress  is not unique: k0s-controller-1
privateAddress  is not unique: k0s-controller-2

Apparently this is fixed in k0sproject/k0sctl#484 and v0.15.2. The project is apparently on 0.15.1, 0.15.2 appears to be a bugfix release for just this one issue, consider upgrading please?