Publish the Katapult provider on the Terraform Registry. Detailed instructions and documentation is available here.

I think the very first version should probably be a pre-release version, e.g. v0.0.1-alpha.0. As detailed here pre-releases are supported, but need to be explicitly specified as the desired provider version. If no version is specified, Terraform will only pick the latest stable version available for a provider.

Some of the resources below will require support within go-katapult, not to
mention additions to the Katapult Core API itself. As such this is issue is
marked as On-Hold for now.

• Add katapult_ssh_key resource with:
  • name attribute
  • public_key attribute
• Add katapult_ssh_key data source.
• Add authorized_keys attribute to katapult_virtual_machine resource
  with:
  • all_organization_keys bool attribute, for adding all organization SSH
    keys to the VM. Default value: true
  • all_user_keys bool attribute, for adding all keys for all users who
    are a member of the organization. Default value: true
  • keys attribute, list of katapult_ssk_key ID strings. When not empty
    set both all_organization_keys and all_user_keys to false.
  • users attribute, list of katapult_user ID strings. When not empty
    set both all_organization_keys and all_user_keys to false.
• Add katapult_user data source.
• Add katapult_users data source.

Add a new katapult_security_group_rules resource to compliment the existing katapult_security_group and katapult_security_group_rule resources.

The purpose of making just the rules list as a whole available as a separate resource, is to make it easy to reuse the same or similar sets of rules across multiple security groups.

Right now you either need to use complex custom locals/vars and dynamic rules in combination with a for_each loop on the katapult_security_group resource. Or you use external_rules and multiple katapult_security_group_rule resources, at the cost of losing absolute control over the full list of rules on the security group.

Links:

• https://pkg.go.dev/github.com/hashicorp/terraform-plugin-mux/tf5to6server
• https://developer.hashicorp.com/terraform/plugin/mux/translating-protocol-version-5-to-6

In order to release a provider on the Terraform Registry, some preparation is needed, full details are available here.

Below is a summary of the steps outstanding:

• Add a license. (#20)
• Provider Documentation, can be generated with the help of tfplugindocs. Fleshing out some example files will probably be needed. (#17)
• Setup automatic release generation with goreleaser and GitHub Actions. This will also include generating a GPG signing key, which will be used to sign all released binaries. This is required as the terraform CLI tool will validate all binaries against the public key published on the Terraform Registry. (#18)
• Add information to the README. (#16)
• Publish the provider on the Terraform Registry. (#19)

I think the very first version should probably be a pre-release version, e.g. v0.0.1-alpha.0. As detailed here pre-releases are supported, but need to explicitly specified as the desired provider version. If no version is specified, Terraform will only pick the latest stable version available for a provider.

Terraform's Plugin SDK was recently superseded by the Plugin Framework:

• https://developer.hashicorp.com/terraform/plugin/framework
• https://github.com/hashicorp/terraform-plugin-framework

Providers can be slowly migrated piece by piece over to the new framework, by hosting both protocol v5 (SDKv2) and v6 (Framework) providers within the same one provider binary:

• https://developer.hashicorp.com/terraform/plugin/mux/translating-protocol-version-5-to-6

Progress

• Setup plugin framework
• Migrate Resources to Framework:
  • katapult_ip
  • katapult_security_group
  • katapult_security_group_rule
  • katapult_virtual_machine
  • katapult_virtual_machine_group
• Migrate Data Sources to Framework:
  • katapult_data_center
  • katapult_disk_template
  • katapult_disk_templates
  • katapult_ip
  • katapult_network_speed_profile
  • katapult_network_speed_profiles
  • katapult_security_group
  • katapult_security_group_rule
  • katapult_security_group_rules
  • katapult_security_groups
  • katapult_virtual_machine
  • katapult_virtual_machine_group
  • katapult_virtual_machine_groups
  • katapult_virtual_machine_package
  • katapult_virtual_machine_packages
• Use Framework for any new Resources and Data Sources.

We currently do not support importing katapult_virtual_machine resources, as
the disk_template and disk_template_options attributes are only relevant
during creation, and are not returned by the API at the moment.

There are plans for the API to return these attributes, but it's not a straight
forward thing, so we probably should not delay import support until it's ready.

Based on the import documentation, we probably want to use
ImportStateVerifyIgnore, and simply require users to define a lifecycle
block with a ignore_changes attribute to avoid the disk template options from
showing as changed after import.

It might be possible to use CustomizeDiff to suppress changes to
attributes which are only available during create. I did have a very brief look
into this back in December, but didn't have much luck, so I'm not sure if it's
possible.

We do not have a license for this project yet. This should be addressed before we make it public.

• Add support for DNS Records in go-katapult.
  (krystal/go-katapult#70)
• Add katapult_dns_zone resource and data source, with:
  • name String attribute.
  • ttl Int attribute.
  • verified read-only Bool attribute.
  • infrastructure_zone read-only Bool attribute.
  • repeatable record attribute, accepting the same attributes as the
    katapult_dns_record resource.
• Add katapult_dns_record resource and data source. This will become a
  little bit tricky, as depending on the type of record, different
  attributes are required. Hence we are going to have to be a little bit
  cleaver about how we structure Terraform's schema to represent this.

A provider published to the Terraform Registry requires documentation. More details are available on Terraform's Provider Documentation docs page.

There's a helper tool called tfplugindocs we should use for this, as it will make fast and easy to generate the initial documentation, and also keeping it up to date moving forward. It looks like some example *.tf files will be needed for each supported resource and data source.

Ideally we should have some sort of docs checking/verifying make target and/or CI task, to ensure that documentation has been re-generated before merging pull requests.

Details about existing networks are available through the API, and the
katapult_ip resource for example can optionally accept a network_id value.
However there's no way to get a network ID through Terraform right now.

Hence we need some data sources to fetch details about networks:

• Add katapult_network data source to return details for a specific
  network by ID or Permalink.
• Add katapult_networks data source to return list of all normal
  non-virtual networks available on the current organization.
• Add katapult_virtual_networks data source to return list of all virtual
  networks available on the current organization.

• Add support for VM groups in go-katapult.
  (krystal/go-katapult#61)
• Add katapult_virtual_machine_group resource.
• Add katapult_virtual_machine_group data source.
• Add katapult_virtual_machine_groups list-style data source.
• Add group_id attribute to katapult_virtual_machine resource.
• Add group_id computed attribute to katapult_virtual_machine data
  source.

Currently the provider requires data_center_id and organization_id arguments. Those values are not exposed by Katapult in any obvious manner. Using the Katapult Core API is simplest way to get those values.

Hence the following changes are needed:

• Rename data_center_id argument to data_center, accepting either the permalink or id value of a data center.
• Rename organization_id argument to organization, accepting either the subdomain or id value of a organization.

The ability to manage Address Lists in Terraform would be very useful.

Address lists can be used as a target for Security Group rules.

Tasks

• Blocking:
  • Address List support in Katapult's Core API.
• Design Terraform resources and data sources once API is in place.

• Add support for disk backup policies in go-katapult.
  (krystal/go-katapult#68)
• Add katapult_disk_backup_policy resource and data source.
• Add katapult_disk_backup_policies data source for listing all backup
  policies.
• Add disk_backup_policy attribute to katapult_virtual_machine resource,
  which takes the same arguments (within reason) as the
  katapult_disk_backup_policy resource.

We want to be able to manage security groups and security group rules through
Terraform.

• Add support for Security Groups to go-katapult:
  • krystal/go-katapult#103
  • krystal/go-katapult#111
• Resources:
  • Add katapult_security_group resource.
    • Has repeatable inbound_rule attribute to define one or more inbound rules.
    • Has repeatable outbound_rule attribute to define one or more outbound rules.
  • Add katapult_security_group_rule resource which allow managing security group rules independently.
• Data Sources:
  • Add katapult_security_group data source to fetch single security group.
  • Add katapult_security_groups data source to fetch list of all security groups.
  • Add katapult_security_group_rule data source to fetch a single security group rule.
  • Add katapult_security_group_rules data source to fetch list of all rules for a single security group.

When krystal/go-katapult#150 is completed, there will be a new "next" version of the go-katapult client, that is generated from Katapult's OpenAPI schema.

We should try switching the new interval/v6provider package over to this new client. This will be the first real-world use of the new client, serving as a good way to evaluate it, and make adjustments to it if needed. While also slowly moving the Terraform provider codebase forward as well.

Populate the README.

Some examples from existing provider projects:

• terraform-provider-google
• terraform-provider-aws
• terraform-provider-azurerm
• terraform-provider-scaleway

Under the hood the provider uses the build_from_spec API endpoint to
create a VM.

We should investigate if it's possible to provide custom buildspec in XML, JSON,
YAML, or HCL, while Terraform behaves normally throughout the rest of the VM
resource's lifecycle, as the spec is only relevant during creation.

Potentially a better approach might be to allow ways to customize specific parts
of the buildspec used during creation if using a full spec proves difficult.

Currently katapult_ip resources can only be imported by their id value. We
also want to support importing them by their address value. This will require
a custom importer function.

Terraform import documentation is available here:
https://www.terraform.io/docs/extend/resources/import.html

Blocked by:

• File Storage Volume support in Katapult's Core API. — Status: Done.
• krystal/go-katapult#136

The ability to manage KEFS / File Storage Volumes in Terraform would be very useful.

Tasks

• Add katapult_file_storage_volume resource, with attributes:
  • id (computed)
  • name
  • data_center (computed)
  • associations (currently, list of VM IDs only)
  • size (in bytes, computed)
  • nfs_location (computed)
• Add katapult_file_storage_volume data source, with attributes:
  • id
  • name
  • data_center
  • associations
  • size (in bytes)
  • nfs_location
• Add katapult_file_storage_volumes data source, returning the list of volumes in the organization.

The katapult_load_balancer resource is currently disabled as load balancer rules are not yet supported, effectively making the laod balancer resource useless.

Hi Team

Reading the documentation avaialble for the Katapult Terraform provider, noticed that currently it is not possible to import resources into Terraform, so that we can manage currently deployed infrastructure. Using katapult_virtual_machine resource as an example, there doesn't appear to be an option to import the resource.

As a comparison, the option to import unmnaged infrastrcuture into Terraform does exist in a resource such as an AWS EC2 Instance:

• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance

Is it possible for this functionality to be added to all the current terraform resources & any new ones going forward?

This issue was created by GitBase for ticket #581.

• Reported by: Matthew Dolman
• Originally reported at: October 26, 2023 08:02
• GitBase Ticket ID: 581
• View ticket on GitBase

The ability to manage Tags in Terraform.

Tasks

• Add katapult_tag resource, with attributes:
  • id (computed)
  • name
  • color
• Add katapult_tag data source, with attributes:
  • id
  • name
  • color
• Add katapult_tags data source to list all tags, with a tags attribute that is a list of tags using the same schema as the katapult_tag data source.

We have a resource and data source implementation in place for load_balancer. It is not exposed and useable in the provider however as load balancer rules are not yet implemented (#49).

As we are migrating to the new Plugin Framework, it makes sense to migrate over the existing load balancer implementation, and implement rules with the new framework afterwards.

For now we've had to disable the tfproviderlint CI check until this issue is fixed:

• bflad/tfproviderlint#255

• Add support for network speed profiles in go-katapult.
  (krystal/go-katapult#80)
• Add katapult_network_speed_profiles data source listing all speed
  profiles.
• Add katapult_network_speed_profile data source to get a single speed
  profile.
• Add network_speed_profile attribute to katapult_virtual_machine
  resource. It must be a Network Speed Profile permalink, and when
  specified, all network interfaces on the VM will be set to use the given
  profile.

The katapult_ip resource acceptance tests only test create and deletion at the
moment. We need some tests which also performs updates against the resource,
changing all attributes which can be changed.

We have the singular katapult_data_center data source already, which fetches details about a specified data center. To complement this, we should also add a plural form which returns a list of all data centers.

This should help improve visibility of data centers, avoiding issues like #81.

Tasks

• Add katapult_data_centers data source, which returns a list of data centers.

I'm trying to use the Katapult Terraform provider, but I cannot figure out what value to put for KATAPULT_DATA_CENTER... I tried LON-1 but it says it's incorrect. I looked everywhere in the docs but can't find a list.

Use retryablehttp to make HTTP requests against the Katapult Core API. It's potentially not needed, but should help ensure the provider is more reliable in the event of any network glitches or the like.

Setup automated releases via GitHub Actions using the steps and examples outlined in Terraform's own documentation here. We will probably want to tweak certain things slightly, but overall the official examples should be a very good starting point.

Tasks:

• Generate a GPG Signing key, storing the private key somewhere secure, and setting it as a repo secret available to GitHub Actions.
• Configure a Release workflow in GitHub Actions based on the examples from Terraform's documentation, using goreleaser to build, sign, and publish the release and binaries onto GitHub.

We want to be able to customize the list of disks and their sizes when creating a new virtual machine. By default machines are given a single system disk with a size determined by the chosen virtual machine package.

The build spec endpoint we use to create virtual machines, supports a override list of system disks. This allows overriding the default system disk behavior, to create one or more system disks with custom sizes.

Example from the API client's buildspec tests: https://github.com/krystal/go-katapult/blob/5a0ac4d078859baa3e69b1cf019a6d8473717812/buildspec/virtual_machine_spec_test.go#L57-L85

As all resources will belong to a specific data center, it makes sense to have a
data_center attribute on all resources.

The value of this resource should be the data center permalink value to keep
things simple.

I'm not sure if the attribute should be read-only, or if we should make it
writable too, allowing the user to specify the data center on a per-resource
basis. Initially my plan was to keep things simple and have the data center be
set globally for the provider, with the intention of using provider aliases for
different data centers.

However I'm coming around to the idea of making things more flexible.

The data center attribute value should always be the data center permalink. If
we make it writable, that should still remain, cause otherwise there will be
issues with state diffing.

Rough task list:

• Add data_center attribute to:
  • katapult_ip resource
  • katapult_ip data source
  • katapult_virtual_machine resource
  • katapult_virtual_machine data source
  • katapult_virtual_machine_group resource
  • katapult_virtual_machine_group data source
• Add acceptance tests to use both uk-lon-01 and us-nyc-01 data centers
  to ensure multi-DC setups work:
  • katapult_ip resource
  • katapult_ip data source
  • katapult_virtual_machine resource
  • katapult_virtual_machine data source
  • katapult_virtual_machine_group resource
  • katapult_virtual_machine_group data source