hashicorp / packer-plugin-yandex Goto Github PK

Please add support custom s3

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

add parameter
storage_access_key_id
storage_secret_access_key

Use Case(s)

Put image to Any storage

Potential configuration

  "post-processors": [
    {
      "type": "yandex-export",
      "folder_id": "b1g8jvfcgmitdrslcn86",
      "subnet_id": "e9bp6l8sa4q39yourxzq",

      "service_account_id": "ajeu0363240rrnn7xgen",

      "paths": [
        "s3://packer-export-bucket/my-exported-image.qcow2",
        "s3://packer-export-bucket/template-supported-get-{{build `ImageID` }}-right-here.qcow2"
      ],
      "keep_input_artifact": true,
      "storage_endpoint": "export.s3.example",
      "storage_access_key_id": "test_key_id".
      "storage_secret_access_key": "test_secret_key",
     
    }
  ]

Potential References

Description

Allows to set a list of security groups on the network interface of the instance.

Use Case(s)

You can limit the list of available network resources during Compute Image build.

Potential configuration

cat main.hcl:

...
security_group_ids = ["enp25gdkdjbr0sna13q1","enp1vnfbcfefghnbubos" ]
...

Potential References

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

I would like to be able to overwrite the existing image option. For example, force_rebuild

Use Case(s)

In a dev environment, it is possible not to generate a new image at each build and to overwrite an existing one for debugging purposes or so. For example in AWS plugin there is an option force_deregister for this

Potential configuration

packer {
  required_plugins {
    yandex = {
      version = "1.0.3"
      source  = "github.com/hashicorp/yandex"
    }
  }
}

source "yandex" "dev" {
  folder_id = "b1g4528l05678l0cak42"
  zone      = "ru-central1-a"

  #Image
  image_name             = "dev-image"
  image_min_disk_size_gb = 10
  force_rebuild             = true

  #Builder
  source_image_family = "ubuntu-2004-lts"

  #Network
  subnet_id    = "e9be0tl4d432ghl0qur3"
  use_ipv4_nat = true
  ssh_username = "ubuntu"

  #Disk
  disk_size_gb = 10
  disk_type    = "network-hdd"

  #Instance
  instance_cores  = 2
  instance_mem_gb = 2
}

build {
  sources = ["source.yandex.dev"]
}

Potential References

AWS option force_deregister - https://www.packer.io/plugins/builders/amazon/ebs#force_deregister

Description

At the moment, it is not possible to configure the vCPU performance levels of the created instance - https://cloud.yandex.com/en/docs/compute/concepts/performance-levels

Use Case(s)

In most cases image creation task is more IO/network bound, so there is no reason to allocate 100% vCPU resources, so it is desirable to allow usage of low vCPU performance levels to optimize costs.

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

Allow use of content of the service account key file in the environment variable YC_SERVICE_ACCOUNT_KEY_FILE in addition to the path to file, similar to what Terraform provider do.

Use Case(s)

Within pipeline stage you can export variable with a content of the key and not create unnecessary file which is a security concern.

Potential References

Similar configuration option in Terraform provider: https://registry.terraform.io/providers/yandex-cloud/yandex/latest/docs#service_account_key_file

Currently the plugin builds images not optimized for deployment. Most if not all images are built for booting VMs so we want those process to be fast. It would be nice if we have an option for that:

source "yandex" "my-custom-image" {
  # ...
  optimized = true
  # ...
}

Overview of the Issue

There is no image family data while use source_image_id as image source to build new image.

Reproduction Steps

Just use source_image_id param and check string

...
==> yandex.this: Using existing SSH private key
==> yandex.this: Using as source image: fd8pr203155555m2b7mf (name: "super-base-g4-202106111", family: "")
==> yandex.this: Use provided subnet id e2ld71sqfzzzzzfsm1
...

Plugin and Packer version

Packer v1.7.1

Overview of the Issue

Variable defaultStorageEndpoint is used to create artifacts of yandex-export, while it can be set in config.
https://github.com/hashicorp/packer-plugin-yandex/blob/b9733757549440a34dc958598c8f7e4cecb5c015/post-processor/yandex-export/post-processor.go#L387C47-L387C47

Reproduction Steps

1. Set storage_endpoint in post-processor "yandex-export"
2. You will get "artifact_id": "https://storage.yandexcloud.net/ always in build artifacts

Plugin and Packer version

yandex-export

Simplified Packer Buildfile

post-processor "yandex-export" {
      endpoint           = "my-custom-endpoint"
      storage_endpoint   = "my-custom-storage-endpoint"

      folder_id          = var.folder_id
      service_account_id = var.service_account_id
      subnet_id          = var.subnet_id
      zone               = var.zone

      use_ipv6 = true

      platform_id  = "standard-v3"

      paths = [
        "${var.s3_path}/${source.name}-{{build `ImageID`}}",
      ]
      keep_input_artifact = true
}

Operating system and Environment details

any

Overview of the Issue

Reproduction Steps

when build with parameter skip_create_image = true

Plugin and Packer version

packer version 1.9.4
yandex plugin version 1.1.3

Simplified Packer Buildfile

source "yandex" "build" {
  zone         = "ru-central1-a"
  folder_id    = var.folder_id
  subnet_id    = var.subnet_id
  token        = var.token

  use_ipv4_nat = true

  source_image_family    = "ubuntu-2204-lts"

  skip_create_image      = true

  platform_id  = "standard-v3"
  disk_type    = "network-ssd"

  ssh_username = "ubuntu"
}

build {
  sources = ["source.yandex.build"]

  provisioner "shell" {
    inline = [
      "echo test"
    ]
  }

}

Operating system and Environment details

hashicorp/packer:latest container from dockerhub

Log Fragments and crash.log files

==> yandex.build: Destroying boot disk...
    yandex.build: Disk has been deleted!
2023/11/30 12:56:48 [INFO] (telemetry) ending yandex.build
==> Wait completed after 2 minutes 57 seconds
2023/11/30 12:56:48 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/11/30 12:56:48 machine readable: yandex.build,error []string{"Failed to find 'image' in state. Bug?"}
==> Builds finished but no artifacts were created.
2023/11/30 12:56:48 [INFO] (telemetry) Finalizing.
Build 'yandex.build' errored after 2 minutes 57 seconds: Failed to find 'image' in state. Bug?

A vulnerability was reported on github.com/dgrijalva/jwt-go v3.2.0. The fix requires the latest release of the go-sdk

Please add the option to select the type of temporary key, as in amazon-ebs
"temporary_key_pair_type": "ed25519"

In new OS versions (in my case centos stream 8), using the RSA key leads to this problem:
ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Details in this issue: hashicorp/packer#11656

Description

As noted in the 7209 issue.

Use Case(s)

Just support new option for image create test purposes.

Potential References

Description

Right now Yandex.Cloud packer plugin uses its own temporary ssh key generation step, but it does not have options for different ssh key types.
Packer builders now can use communicator.StepSSHKeyGen step from packer-plugin-sdk package, which have all the required options.

Use Case(s)

Fedora 33/34 builds, which do not accept ssh-rsa keys anymore

im currently working on PR

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

It possible to configure Yandex API endpoint, but storage endpoint bind to the constant in the source code, is it possible to allow customization of storage and region via configuration or, if supported by Yandex API, make it configurable via API calls? For first case I've modest patch, so would gladly post it as PR, if acceptable by code maintainers.

Use Case(s)

Use of Yandex packer plugin in custom/nonstandard environments with different storage endpoint and region settings.

Potential configuration

storage_endpoint: "storage.north-3.yandex-cloud.net"
storage_region: "ru-north-3"

Overview of the Issue

Right now in order to upload an image (like previously brewn qcow2 file) you have to either enable public access on your bucket, or come up with a more-or-less complicated set of ACLs around it.

Without that, image cannot be created because Compute ImageService.Create has no permissions to the object.
One way out of this situation is to use URL Presigning, which BTW is already used in case of importing other artifacts already present in Object Storage.

To be honest, I'm not really sure if this is a bug, but right now it definitely makes working with YC significantly more involved that it potentially could be.

Reproduction Steps

1. Create an empty bucket with default permissions.
2. Point packer to that bucket, with whatever file that ends with .qcow2
3. Receive an error after Compute launches their op: rpc error: code = InvalidArgument desc = url source not found

Plugin and Packer version

From 1.1.2

Simplified Packer Buildfile

source "file" "base" {
    source = "testing.qcow2"
    target =  "testing.qcow2"
}

build {
    sources = ["sources.file.base"]

    post-processors {
        post-processor "yandex-import" {
            service_account_key_file = "key.json"
            folder_id = "b48..."
            service_account_id = "f08..."
            bucket = "kuzpactor-service"
        }
    }
}

Operating system and Environment details

MacOS, 13.4.1 (22F82), on Apple Silicon (M1).
The plugin is launched with patch from #83, because otherwise the file ends up in the wrong installation.

Log Fragments and crash.log files

==> file.base (yandex-import): Uploading file testing.qcow2 to bucket kuzpactor-service/packer-import-1692557311.qcow2...
==> file.base (yandex-import): Source of Image creation: object source, url: https://storage.il.nebius.cloud/kuzpactor-service/packer-import-1692557311.qcow2
==> file.base (yandex-import): Creating Yandex Compute Image test-image-1692557311 within operation "alk9aaj99asv7v41qak6"
==> file.base (yandex-import): Waiting for Yandex Compute Image creation operation to complete...
2023/08/20 20:48:47 packer-plugin-yandex plugin: 2023/08/20 20:48:47 error: failed to create Yandex Compute Image: operation (id=alk9aaj99asv7v41qak6) failed: rpc error: code = InvalidArgument desc = url source not found
2023/08/20 20:48:47 [INFO] (telemetry) ending yandex-import