Code Monkey home page Code Monkey logo

helm-mapkubeapis's Introduction

Helm mapkubeapis Plugin

License Go Report Card CircleCI Release

mapkubeapis is a Helm v3 plugin which updates in-place Helm release metadata that contains deprecated or removed Kubernetes APIs to a new instance with supported Kubernetes APIs, or entirely removes references to resources that use APIs that were removed and do not have a successor. Jump to background to the issue for more details on the problem space that the plugin solves.

Note: Charts need to be updated also to supported Kubernetes APIs to avoid failure during deployment in a Kubernetes version. This is a separate task to the plugin.

Prerequisite

  • Helm client with mapkubeapis plugin installed on the same system
  • Access to the cluster(s) that Helm manages. This access is similar to kubectl access using kubeconfig files. The --kubeconfig, --kube-context and --namespace flags can be used to set the kubeconfig path, kube context and namespace context to override the environment configuration.
  • If you try and upgrade a release with unsupported APIs then the upgrade will fail. This is ok in Helm v3 as it will not generate a failed release for Helm.
  • A mapping file is used to define the API mappings. By default, the strings in the mapping file contain UNIX/Linux line feeds. This means that \n is used to signify line separation between properties in the strings. This should be changed if the Helm release metadata is rendered in Windows or Mac. Refer to API Mapping for more details.
  • The plugin updates the lastest release version. The latest release version should be in a deployed state as you want to update a successful deployment. If it is not then you need to delete the latest release version. The command to remove a release version is:
    • Helm v3: kubectl delete configmap/secret sh.helm.release.v1.<release_name>.v<latest_version_number> --namespace <release_namespace>

Install

Based on the version in plugin.yaml, release binary will be downloaded from GitHub:

$ helm plugin install https://github.com/helm/helm-mapkubeapis
Downloading and installing helm-mapkubeapis v0.1.0 ...
https://github.com/helm/helm-mapkubeapis/releases/download/v0.1.0/helm-mapkubeapis_0.1.0_darwin_amd64.tar.gz
Installed plugin: mapkubeapis

For Windows (using WSL)

Helm's plugin install hook system relies on /bin/sh, regardless of the operating system present. Windows users can work around this by using Helm under WSL.

$ wget https://get.helm.sh/helm-v3.0.0-linux-amd64.tar.gz
$ tar xzf helm-v3.0.0-linux-amd64.tar.gz
$ ./linux-amd64/helm plugin install https://github.com/helm/helm-mapkubeapis

Usage

Map Helm deprecated or removed Kubernetes APIs

Map release deprecated or removed Kubernetes APIs in-place:

$ helm mapkubeapis [flags] RELEASE 

Flags:
      --dry-run                  simulate a command
  -h, --help                     help for mapkubeapis
      --kube-context string      name of the kubeconfig context to use
      --kubeconfig string        path to the kubeconfig file
      --mapfile string           path to the API mapping file (default "config/Map.yaml")
      --namespace string         namespace scope of the release

Example output:

$ helm mapkubeapis cluster-role-example --namespace test-cluster-role-example         
2022/02/07 18:48:49 Release 'cluster-role-example' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2022/02/07 18:48:49 Get release 'cluster-role-example' latest version.
2022/02/07 18:48:49 Check release 'cluster-role-example' for deprecated or removed APIs...
2022/02/07 18:48:49 Found 1 instances of deprecated or removed Kubernetes API:
"apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
"
Supported API equivalent:
"apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
"
2022/02/07 18:48:49 Found 1 instances of deprecated or removed Kubernetes API:
"apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
"
Supported API equivalent:
"apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
"
2022/02/07 18:48:49 Finished checking release 'cluster-role-example' for deprecated or removed APIs.
2022/02/07 18:48:49 Deprecated or removed APIs exist, updating release: cluster-role-example.
2022/02/07 18:48:49 Set status of release version 'cluster-role-example.v1' to 'superseded'.
2022/02/07 18:48:49 Release version 'cluster-role-example.v1' updated successfully.
2022/02/07 18:48:49 Add release version 'cluster-role-example.v2' with updated supported APIs.
2022/02/07 18:48:49 Release version 'cluster-role-example.v2' added successfully.
2022/02/07 18:48:49 Release 'cluster-role-example' with deprecated or removed APIs updated successfully to new version.
2022/02/07 18:48:49 Map of release 'cluster-role-example' deprecated or removed APIs to supported versions, completed successfully.

API Mapping

The mapping information of deprecated or removed APIs to supported APIs is configured in the Map.yaml file. The file is a list of entries similar to the following:

 - deprecatedAPI: "apiVersion: extensions/v1beta1\nkind: Deployment"
    newAPI: "apiVersion: apps/v1\nkind: Deployment"
    deprecatedInVersion: "v1.9"
    removedInVersion: "v1.16"

The plugin when performing update of a Helm release metadata first loads the map file from the config directory where the plugin is run from. If the map file is a different name or in a different location, you can use the --mapfile flag to specify the different mapping file.

The OOTB mapping file is configured as follows:

  • The search and replace strings are in order with apiVersion first and then kind. This should be changed if the Helm release metadata is rendered with different search/replace string.

  • The strings contain UNIX/Linux line feeds. This means that \n is used to signify line separation between properties in the strings. This should be changed if the Helm release metadata is rendered in Windows or Mac.

  • Each mapping is composed of:

    • The original API group and version (required);
    • The new API group and version (optional);
    • The Kubernetes version that the API is deprecated in (optional); and
    • The Kubernetes version that the API is removed in (required).

    This information is important as the plugin checks that the deprecated version (or the removed version, when deprecated version is unset) is later than the Kubernetes version that it is running against. If it is then no mapping occurs for this API as it not yet deprecated in this Kubernetes version and hence the new API is not yet supported. Otherwise, the mapping can proceed.

    When the new API group is unset, the mapping is assumed to be a removal of an API for which there is no successor. In this scenario, all the resources that refer to the removed API are entirely removed from the release metadata. This aims to address scenarios where the API was replaced with a different mechanism that does not take the same input format, such as the removal of the PodSecurityPolicy API.

Note: The Helm release metadata can be checked by following the steps in:

Background to the issue

For details on the background to this issue, it is recommended to read the docs appropriate to your Helm version. The docs can be accessed as follows:

The Helm documentation describes the problem when Helm releases that are already deployed with APIs that are no longer supported. If the Kubernetes cluster (containing such releases) is updated to a version where the APIs are removed, then Helm becomes unable to manage such releases anymore. It does not matter if the chart being passed in the upgrade contains the supported API versions or not.

This is what the mapkubeapis plugin resolves. It fixes the issue by mapping releases which contain deprecated or removed Kubernetes APIs to supported APIs. This is performed inline in the release metadata where the existing release is superseded and a new release (metadata only) is added. The deployed Kubernetes resources are updated automatically by Kubernetes during upgrade of its version. Once this operation is completed, you can then upgrade using the chart with supported APIs.

Helm v2 Support

Helm v2.17.0 was the final release of Helm v2 in October 2020. Helm v2 is unsupported since November 2020, as detailed in Helm 2 and the Charts Project Are Now Unsupported. mapkubeapis Helm v2 support finished in release v0.2.0.

Developer (From Source) Install

If you would like to handle the build yourself, this is the recommended way to do it.

You must first have Go v1.18+ installed, and then you run:

$ mkdir -p ${GOPATH}/src/github.com
$ cd $_
$ git clone [email protected]:helm/helm-mapkubeapis.git
$ cd helm-mapkubeapis
$ make
$ export HELM_LINTER_PLUGIN_NO_INSTALL_HOOK=true
$ helm plugin install <your_path>/helm-mapkubeapis

That last command will use the binary that you built.

helm-mapkubeapis's People

Contributors

bernermic avatar brandond avatar branttaylor avatar dependabot[bot] avatar fedterzi avatar hickeyma avatar hydeenoble avatar jhagrid77 avatar jsoref avatar jthornec avatar mattclement avatar mattfarina avatar n4rm0 avatar nvanheuverzwijn avatar pgier avatar tchernomax avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

helm-mapkubeapis's Issues

Plugin returns error: gzip: stdin: not in gzip format

Since this morning our builds are failing. The tar.gz seems not to exist?

Downloading and installing helm-mapkubeapis v0.4.1 ...
https://github.com/helm/helm-mapkubeapis/releases/download/v0.4.1/helm-mapkubeapis_0.4.1_linux_amd64.tar.gz
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Error: plugin install hook for "mapkubeapis" exited with error

We install the plugin via:

helm plugin install https://github.com/helm/helm-mapkubeapis

Deployment from Helm 2 to Helm3 is not correct mapped

I have a kube Cluster 1.17 with old helm2 deployments.
During migration to helm, i've added mapkupeapis to helm3 instance. But some mappings was not done successful, currently because of unknow reason:

Output from helm command:
image

Output from release.yaml after running mapkubeapi command:
image

Any hint how to debug this?

Helm 2 Support

It would be great to have Helm 2 support for this plugin.

Thanks!

Did not find any deprecated APIs

I'm using HELM 2.16.7. HELM was not able to make the update: Error: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

I used your tool, but it did not find any problem in the last deployment.
2020/05/06 14:35:39 Finished checking release 'icms-uat' for deprecated or removed APIs.
2020/05/06 14:35:39 Release 'icms-uat' has no deprecated or removed APIs.

Could you pls check what can be the problem?

Update/create timing window

There is a problem with this plugin as it depends on two successful etcd backed operations to correctly map a release (update, create). If the process dies after the old release is updated to StatusSuperseded [1] but before creating the new revision[2], this release is toast as there aren't any Deployed revisions. This can also happen if etcd is having problems and the cfg.Releases.Create call fails.

I wonder if this command could be updated to check for a release with no Deployed revisions and handle accordingly? That would solve both cases without having too much special case handling.

[1] https://github.com/hickeyma/helm-mapkubeapis/blob/master/pkg/v3/release.go#L67
[2] https://github.com/hickeyma/helm-mapkubeapis/blob/master/pkg/v3/release.go#L82

Darwin arm64?

Hello.

Do you have a plan to support native binary for Apple Silicon M1?

Multi-namespace release is not fully upgraded

A silent K8s upgrade from Google Cloud broke my helm charts, since they still use deprecated APIs, namely

kind: DaemonSet
apiVersion: extensions/v1beta1

and

kind: Deployment
apiVersion: apps/v1beta1

Using the plugin the Deployments were upgraded successfully, but the DaemonSet is located in a different K8s namespace.

What happened

helm outputs:

$ helm ls
NAME             	NAMESPACE	REVISION	UPDATED                             	STATUS  	CHART              	APP VERSION
PROJECT	default  	65      	2020-12-09 23:35:36.911248 +0100 CET	deployed	PROJECT-2.13.3

Commands run:

$ helm mapkubeapis PROJECT
2020/12/09 23:48:56 Release 'PROJECT' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2020/12/09 23:48:56 Get release 'PROJECT' latest version.
2020/12/09 23:48:56 Check release 'PROJECT' for deprecated or removed APIs...
2020/12/09 23:48:56 Finished checking release 'PROJECT' for deprecated or removed APIs.
2020/12/09 23:48:56 Release 'PROJECT' has no deprecated or removed APIs.
2020/12/09 23:48:56 Map of release 'PROJECT' deprecated or removed APIs to supported versions, completed successfully.

$ helm upgrade PROJECT ./helm/prod --dry-run
Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "DaemonSet" in version "extensions/v1beta1"

What I expect to happen

DaemonSet in kube-system namespace should be updated as well.

Versions and K8s config

Versions:

$ helm version
version.BuildInfo{Version:"v3.4.2", GitCommit:"23dd3af5e19a02d4f4baa5b2f242645a1a3af629", GitTreeState:"dirty", GoVersion:"go1.15.5"}

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.13-gke.404", GitCommit:"81e2afbf51b876d15f0af546998c670207fdd592", GitTreeState:"clean", BuildDate:"2020-10-30T23:52:44Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}

DaemonSet yaml:

kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system  # helm deployment is in default namespace
  labels:
    k8s-app: traefik-ingress-lb
spec:
  selector:
    matchLabels:
      name: traefik-ingress-lb
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:{{ .Chart.Version }}
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80

Issue with NetworkPolicy

Hello,

After launching the plugin v0.0.14 to remove deprecated api, I encountered the following error when performing hem upgrade.

$ helm upgrade --install releaseName --dry-run
Error: UPGRADE FAILED: unable to build kubernetes objects from current release manifest: error parsing : error converting YAML to JSON: yaml: line 3: mapping values are not allowed in this context

After some investigation I found out the network policy object is incorrectly mapped during the api replacement.

$ helm get manifest releaseName
[...]
# Source: chart/templates/networkpolicy.yaml
**apiVersion: apiVersion: networking.k8s.io/v1**  
kind: NetworkPolicy
[...]

The field apiVersion is incorrectly set in helm and breaks the release.

Feature Request: Support reversed kind and apiVersions

Sometimes I run into kind and apiVersion specified in the opposite order
The fix is to add the reversed one in Map.yaml i.e.

  • deprecatedAPI: "kind: Deployment\napiVersion: extensions/v1beta1"
    newAPI: "apiVersion: apps/v1\nkind: Deployment"
    deprecatedInVersion: "1.9"
    removedInVersion: "1.16"

It would be cool if this was supported automatically.
Thanks for the great work on this plugin!

Mapkubeapis doesn't update Deployments api

Using Helm3, k8s 1.16

 helm mapkubeapis dinghy-ping --namespace default
2020/07/30 10:45:16 Release 'dinghy-ping' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2020/07/30 10:45:16 Get release 'dinghy-ping' latest version.
2020/07/30 10:45:17 Check release 'dinghy-ping' for deprecated or removed APIs...
2020/07/30 10:45:17 Found deprecated or removed Kubernetes API:
"apiVersion: extensions/v1beta1
kind: Ingress"
Supported API equivalent:
"apiVersion: networking.k8s.io/v1beta1
kind: Ingress"
2020/07/30 10:45:17 Finished checking release 'dinghy-ping' for deprecated or removed APIs.
2020/07/30 10:45:17 Deprecated or removed APIs exist, updating release: dinghy-ping.
2020/07/30 10:45:17 Set status of release version 'dinghy-ping.v134' to 'superseded'.
2020/07/30 10:45:17 Release version 'dinghy-ping.v134' updated successfully.
2020/07/30 10:45:17 Add release version 'dinghy-ping.v135' with updated supported APIs.
2020/07/30 10:45:17 Release version 'dinghy-ping.v135' added successfully.
2020/07/30 10:45:17 Release 'dinghy-ping' with deprecated or removed APIs updated successfully to new version.
2020/07/30 10:45:17 Map of release 'dinghy-ping' deprecated or removed APIs to supported versions, completed successfully.

However, it fails to update the Deployment object in manifests.

helm get manifest dinghy-ping -n default | grep -B1 -A2 extension
# Source: dinghy-ping/templates/deployment.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dinghy-ping

And as expected helm upgrade command will still fail because deprecated API still in helm manifests.

Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"
helm.go:84: [debug] unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

I confirmed Map.yaml contains the depreciated API for "Deployments" that should be replaced. However, mapkubeapis doesn't seem to be updating this for the release.

Not all apis removed from release

Hello.
We are using helm v3.7.2 + mapkubeapis v0.4.1
k8s node v1.24.9 (where old release version deployed) and v1.25.6 (where plugin executed)

On some releases plugin removes all removed apis, but one of this has removed only 4 of 5 apiVersion: policy/v1beta1\nkind: PodSecurityPolicy\n

There are logs of that execution:

/tmp $ helm-v3.7.2 mapkubeapis project-monitoring --namespace project 
2023/04/04 21:09:34 Release 'project-monitoring' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2023/04/04 21:09:34 Get release 'project-monitoring' latest version.
2023/04/04 21:09:34 Check release 'project-monitoring' for deprecated or removed APIs...
2023/04/04 21:09:34 Found 4 instances of deprecated or removed Kubernetes API:
"apiVersion: policy/v1beta1
kind: PodSecurityPolicy
"
No supported API equivalent
2023/04/04 21:09:34 Finished checking release 'project-monitoring' for deprecated or removed APIs.
2023/04/04 21:09:34 Deprecated or removed APIs exist, updating release: project-monitoring.
2023/04/04 21:09:34 Set status of release version 'project-monitoring.v1' to 'superseded'.
2023/04/04 21:09:35 Release version 'project-monitoring.v1' updated successfully.
2023/04/04 21:09:35 Add release version 'project-monitoring.v2' with updated supported APIs.
2023/04/04 21:09:35 Release version 'project-monitoring.v2' added successfully.
2023/04/04 21:09:35 Release 'project-monitoring' with deprecated or removed APIs updated successfully to new version.
2023/04/04 21:09:35 Map of release 'project-monitoring' deprecated or removed APIs to supported versions, completed successfully.

I used the following command to get secret manifests:
kubectl get secret -n project sh.helm.release.v1.project-monitoring.v1 --template={{.data.release}} | base64 -d | base64 -d | gzip -d | jq -r .manifest > before.json

Please find before.yaml and after.yaml attached. I left only the beginning of the files where there are differents

May be it will useful information: release where plugin removed all three 'wrong' apis has exported secret size 71kb, but release where removed 4 of 5 has exported secret size 477kb and 3.2mb of decoded unzipped manifest field.

issues executing with "oidc" auth provider

Auth pro

2023/12/04 16:04:10 Release 'wp3-ms-dep' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2023/12/04 16:04:10 Get release 'wp3-ms-dep' latest version.
Error: failed to get release 'wp3-ms-dep' latest version: query: failed to query with labels: no Auth Provider found for name "oidc"
Error: plugin "mapkubeapis" exited with erro

mapkubeapis failing on helm version "v3.4.2"

Hi
I was delighted to find this tool, but for some reason it doesn't work for me.
Can you please advise?
Thanks
Ariela

helm mapkubeapis rel-name --namespace ns-name --dry-run

2023/06/19 15:27:17 NOTE: This is in dry-run mode, the following actions will not be executed.
2023/06/19 15:27:17 Run without --dry-run to take the actions described below:
2023/06/19 15:27:17
2023/06/19 15:27:17 Release 'rel-name' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2023/06/19 15:27:17 Get release 'rel-name' latest version.
Error: failed to get release 'rel-name' latest version: query: failed to query with labels: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"
Error: plugin "mapkubeapis" exited with error

No Auth Provider Found

I am trying to use mapkubeapis and am receiving the following error:

helm mapkubeapis --dry-run --namespace <namespace> --kubeconfig /Users/nat-ray/.kube/config --kube-context <context-name>  <helm-chart-name>
2023/04/13 11:01:52 NOTE: This is in dry-run mode, the following actions will not be executed.
2023/04/13 11:01:52 Run without --dry-run to take the actions described below:
2023/04/13 11:01:52
2023/04/13 11:01:52 Release 'rabbitmq-bitnami' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2023/04/13 11:01:52 Get release 'helm-chart-name' latest version.
Error: failed to get release 'helm-chart-name' latest version: query: failed to query with labels: no Auth Provider found for name "oidc"
Error: plugin "mapkubeapis" exited with error
  1. All of my other helm commands work well.
  2. All of my kubectl commands in this context work well.
  3. I have tried setting the KUBECONFIG environment variable and running with the --kube-context flag, but the result is the same.
  4. I have tried updating the plugin using helm plugin update mapkubeapis.
  5. I have tried uninstalling and reinstalling the plugin.

Is there something I am missing? Thank you.

Release has no deprecated or removed APIs

This helm release fails to upgrade with this message:

Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

But running a helm mapkubeapis XXX -n kube-system does not find any APIs that are at issue:

2020/05/07 11:30:48 Release 'XXX' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2020/05/07 11:30:48 Get release 'XXX' latest version.
2020/05/07 11:30:49 Check release 'XXX' for deprecated or removed APIs...
2020/05/07 11:30:49 Finished checking release 'XXX' for deprecated or rempved APIs.
2020/05/07 11:30:49 Release 'XXX' has no deprecated or removed APIs.
2020/05/07 11:30:49 Map of release 'XXX' deprecated or removed APIs to supported versions, completed successfully.

Versions:

  • Kubernetes 1.16.9
  • Helm 3.2.0
  • Helmfile 0.114.0
  • mapkubeapis 0.0.9

Side note, I also noticed a typo in the output: rempved

API not detected with `-dirty` server version

I am running Openshift/OKD and have the following kubernetes version.

Server Version: 4.12.0-0.okd-2023-03-18-084815
Kubernetes Version: v1.25.0-2786+eab9cc98fe4c00-dirty

Unfortunately this version is not detected as v1.25.0 and the plugin will not report and fix APIs that are removed in 1.25.0.

Kube mapping value not detected

Hi folks 👋

I would like to raise you a recent issue I have with this amazing tool.
Indeed I noticed recently that I wasn't able to update deprecated resource in my helm releases

By looking at the code and making some try I finally came to the conclusion that the comparison between the modifiedManifest and the deprecatedAPI failed every time since the modifiedManifest yaml contains double quotes (") for each values and the map in the config file don't : so the plugin never try to apply the update on the resource 😅

Here is the line of the comparison

Here is a sample of the modifiedManifest when I print it through the code

apiVersion: "networking.k8s.io/v1beta1"
kind: "Ingress"

Also I noticed this is not consistent for all resources, for example we have custom resources called DNSEndpoint (from the external-dns project)

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint

When I modify the map file and I manually add the escaped double quote, it works:

 - deprecatedAPI: "apiVersion: \"networking.k8s.io/v1beta1\"\nkind: \"Ingress\"\n"
    newAPI: "apiVersion: \"networking.k8s.io/v1\"\nkind: \"Ingress\"\n"
    deprecatedInVersion: "v1.19"
    removedInVersion: "v1.22"

Maybe some kind of sanitization is required here I don't know, let me know 😉

I'm not sure when it starts to do that, so here is my config:

  • kube client v1.26.3
  • kube server v1.23.16
  • helm v3.11.2
  • helm plugin mapkubeapis 0.4.1

Don't hesitate to reach me if you need more details
Cheers 😉

mapkubeapis plugin not finding release

Hi!
mapkubeapis plugin is not finding release "gitlab"

Helm version

helm2 version
...
Client: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}

mapkubeapis version

$ helm2 plugin list | grep mapkubeapis        
...
mapkubeapis     0.0.15  Map release deprecated Kubernetes APIs in-place

List releases in namespace imp-gitlab

$ helm2 --kube-context=imp-hhe-prd --namespace imp-gitlab list
...
NAME    REVISION        UPDATED                         STATUS          CHART           APP VERSION     NAMESPACE 
gitlab  13              Thu May 21 13:44:06 2020        DEPLOYED        gitlab-2.0.3    master          imp-gitlab

List secrets in namespace kube-system

$ kubectl --context imp-hhe-prd -n kube-system get secret|grep gitlab
gitlab.v10                                           Opaque                                1      553d
gitlab.v11                                           Opaque                                1      518d
gitlab.v12                                           Opaque                                1      356d
gitlab.v13                                           Opaque                                1      356d
gitlab.v4                                            Opaque                                1      673d
gitlab.v5                                            Opaque                                1      673d
gitlab.v6                                            Opaque                                1      673d
gitlab.v7                                            Opaque                                1      587d
gitlab.v8                                            Opaque                                1      574d
gitlab.v9                                            Opaque                                1      553d

Running mapkubeapis

$ helm2  mapkubeapis --kube-context imp-hhe-prd --namespace kube-system --v2 gitlab
...
2021/05/12 13:52:53 Release 'gitlab' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2021/05/12 13:52:53 Get release 'gitlab' latest version.
Error: Failed to get release 'gitlab' latest version: release: "gitlab" not found
Error: plugin "mapkubeapis" exited with error

Any idea of what is going on?
How can i debug this issue?

Thank you!

kube version has a plus in the minor version string

The response from kubeVersion.Minor here will be e.g 16+ rather than 16. This means that strconv.ParseFloat fails, as 1.16+ cannot be properly parsed.

I'm creating an issue because I'm not certain of the best way to tackle this – do we just want to drop the +? Should we rely on GitVersion instead?

The advantage of GitVersion would be that we can do a proper semver compare, as versions like v1.16.6-beta.0 or v1.16.8-gke.15 are both valid SemVer versions, and can be properly compared against e.g "1.9". This would also make the Map.yaml a bit more readable as we could do "1.9" rather than "1.09" which looks a bit weird (after all, it is version 1.9).

Happy to PR this once there's some decision on how we want to do the version comparison :)

How to upgrade release pointing to beta app when release doesn't exist

I am making a brand new release with upgrade. Normally, it should work, but I am getting the classic:

Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta1"

In my situation, it is impossible to change the kubernetes install (apparently on or before 1.6) or the helm version (2.14.3) So I am trying to use this plugin to not use the v1beta1 app. Fixes to this error seem arcane and esoteric... but this codebase seems to be designed to fix this issue. The issue is, I do not have a "Helm release" to pass as an argument to "mapkubeapis" - because I get the above error when trying to make the release. It seems fixing the issue depends on having a release, and having a release depends on fixing the issue.

I also can't change any yaml files or anything.

Here is what I'm trying to do (order does not matter):

helm --tiller-namespace=${PRODUCT_CODE} upgrade -i ${HELM_RELEASE_NAME} ${CHART_REPO}
--version ${CHART_VERSION}
--namespace=${PRODUCT_CODE} -f helm/common/values.yaml -f helm/${CLUSTER}/values.yaml
--set image.tag=${IMAGE_TAG}
--set ingress.fqdn=${INGRESS_FQDN}
--set deploytime=${DEPLOYTIME}

helm mapkubeapis ${HELM_RELEASE_NAME} --namespace ${PRODUCT_CODE} --v2

Please help out a noob... Thanks

Ability to update chart templates

Hi,
maybe its possible to also update self created charts with this plugin?
For example to just pass a --chart parameter and a path and then the template files will be adjusted?

What do you think?

Best regards
eloo

Failing on plugin installation

getting error like below

Helm 3.7.1 and 3.8.1 used but same issue

./linux-amd64/helm plugin install https://github.com/helm/helm-mapkubeapis
Downloading and installing helm-mapkubeapis v0.3.0 ...
https://github.com/helm/helm-mapkubeapis/releases/download/v0.3.0/helm-mapkubeapis_0.3.0_linux_amd64.tar.gz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Error: plugin install hook for "mapkubeapis" exited with error

Add support for rbac.authorization.k8s.io API changes

When trying to migrate kube2iam chart from 2.0.1 to 2.1.0+, got similar errors as noted in helm/helm#7219

This tool was able to help us patch the release for the DaemonSet apiVersion change to apps/v1 (thanks!) but unable to fix the APIs for the ClusterRole and ClusterRoleBinding.

I'm not sure of the wider consequences thereof, but seems this tool could also potentially help map
rbac.authorization.k8s.io/v1beta1
to
rbac.authorization.k8s.io/v1

This would presumably also affect Role and RoleBinding.

Example chart change to assist with testing.
helm/charts@0361720#diff-93cdf1f4ea0c4b64a41c7ff4a84a60c2

Substring APIs are matched too agressively using the default config

I have a Helm config which contains deprecated instances of both

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole

and

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding

Edit: The default mapping contains "apiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole" which matches/replaces both of the above types. And when the plugin checks for ClusterRoleBinding, it doesn't find any because they have already been replaced by the previous match.

So I was confused when running a dry-run and mapkubeapis was only finding ClusterRole and not ClusterRoleBinding.
Digging into the code, I found the reason for this is because ClusterRole is a substring of ClusterRoleBinding and all instances of ClusterRoleBinding had been replaced before it was checked. In this case this is only an issue of CLI output, and the resulting helm manifest ends up the same either way. However, I can see a situation where this could cause a problem where you want to upgrade one kind but not the other.

Some possible fixes would be to:
(1) Change the order of matches in the default config so that longer kinds come first
(2) add newline characters at the end of all the map strings in the mapping config
(3) use a regex instead of strings.Replace() which matches the strings more strictly with maybe any whitespace at the end

Issue with Helm2 tillerless on Rancher

Support for Rancher installed applications would be great, as it seems they're doing something different to regular tillerless for their Helm 2 installs.

An example Rancher release from one of our clusters - not one that needs to be patched but rather one that displays the issue without including any secrets;

$ kubectl --namespace=aleol57 get cm kube-ops-view.v3 -o yaml
apiVersion: v1
data:
  release: H4sIAAAAAAAC/+xbzY8kyVXXfO2MAyMtjYXECovnalvabU9mfUxPd09KIzE7MzaNvbOt6Zmx0Gq1E5X5qiqmIzPSEZHVU+4tJISEfEDygauBA0j8EUgc7AsHOHC0kJE4wAGEuGCBJRAoIjIrP+pjqnpm7ZV36tCqiox48T5+78WLl6/JL59kffREqrwxw9Otn14if3Pp2oXWDy89EkDDEJUCPUL4RtZHmaBGBe+nKKlmIqEcnjA8hceHASFdH77GpNKgNJXarjGUQ80hleL5JCAEoD5ESM+HB+IURIqJXTEQnItTlgzh8cNvAktgIjIJfSlOFUpHYaR1GrTbXISUj4TSwUGn023TlLXH3bYl205ojCqlIao25Sj4zf22QjlmZqAmbZuQr0kRg8i0YhFaFkKeKY3yuv1hlqF0zCi7f759QcE/menFZ9rnLPMVtgk54kgVgpYTkMgFjYxMhmJKhwhsYAQDhQitY8fZ44SOKeO0zxHakAjAJEoFS7SC8sFASMglaV2HVEQQSrSmgJgNRxo0PUGgEIsYE+2Trc9f+6O//4d/vvLmv//3j/7w0lufv/ajv/7Bf1158/v/8cffu7Dz5uN0KGmEEIo45ajxrb/9FvnXi01AvG0kVkG7PWR6lPX9UMTt0TMain5DmW+tPbN1pev3/M7Ow1Wg8kAijTyR8AmoidIYQ0TVqC+ojKwi4oxrlnKEbxyowmqqR0qD9D43W9AjoqCvgi+SqzlbW1sjTBKWDH/L/e76Eb57ZSh0yn/ncSGNpKe+kyhTKEORaKPbpcK1Y2o4qQ96XAyFnybDhxfH3Wdv9Dr+rt/Z+sll8qsa45RTjar94P1H9499/Vxv/fjyL4TvnZ2B/xCtI/gPimGYTktnPDuDQn5o1RTmDzLODa0W+GYJIWdnxm38J5RnxteSoUSlfEyMY0TgTafncuXFVDVXMJ2qszPjhTCdBm0rTGOW0YBhEabTBU9TqkcwnVrGDRXD4ackKvzgGvm1EncfjZCnKJWvU771l9fOzto7MGZxAAo1DBhHPUnxdpwpTcMRBrDTnk6JmUXuP09pElkBrB7EwKl9RKX2ST7PgwgHLJkzsDOuV86hGdfg37WLDVxmGjUz3x+jlMauH4OWWRLC3g37lcXH2WDAnkPLK4kV6nZc3jXKMPIXexhoTeDbGeVswDACmqaWf598Cx11O1+bPYwsCvoY0kwhKBHXvNFKPWDIIwVUInAWM40RaAF6xBS83Z9Yjdx7cGzmGmurFMN3fHI4MAiwaLBETFihLFFOeW6MaThlnEMfIVOGTwXUMp9z+wIFlx5U6KWC9OLhTK/FnKUT1lI8VyWlL1shgtvr27bC50wfjkotjpS81kY3ZjCVLNEDaH1FeV9RrQY1t+8maFv2vYbCinmN64xRKuOnVDkT53hxszjtI3+hme3c1lKhqjp335/ke34MElNuQnLrqy1ofdTaSNqtH14hb5VBJA+1UnD0JzTmW39xpYE42aehHzotGAI0ZTknAdhnNNMjIdl3bOTyTw6Uz0R73O2jpl1ywpIogLtul4eCI4lR04hqGhBwmlLmGxhvriVlwh5JAZhIn4Q8i5ZEInPMWAIj5LGvRm2r2FXrcs3PFs7vHNOEDjHy+hNLZ4awPLovX8gSpWkSYn3ZA3fYLFkzLrRpljhT30nTwtpLl6VUak8Mlu00U956BzWRGUcVEA9oyr4uRZaqAD5otT4kABKVyGSIdiQREarWdWilIlL28Rhl3z7iTOnWh00SMWrJQpUDYyOCQ9TmQU63huI/f4N8aSGK+ywxp7MD8/9+wmB+1232GtOfUkwLjg9xYGxRQHKFmQnAfLTafFOV9Z9hqC0CvJxiruM7YSiyRFvhNyVbrLHJ+LyCihy97iV/9QXy66WXRJhyMbG5pPWOP/lCFf40TVV7XID83mxuDdxrwtfy+9oRPiFHMKloYMNoyllIlZtVRDg3eNcAzdFXyDHUQjpLxFSHo29WTPMKjHMOVy5QmTNVgZj58Bp/r4TDl4DQS8HovFB6CTidyyDu0zguUxE5qLjIAlr8Lo35gscfA0siTDQcNKm5i7gdKnC7YCM6GLCE6UkFUflIsGDf2exF2zY2ndvK5BvHuUuUk6qji7asrTrPtlrwopBVzqkMLtq0umbtPU+ZHlUtdIxhJpme3BWJxue6nKzqD2oMrLedqh1rD2YuuTjfstWW9U67szN345tOW9XQZojN75nPqnGWX0NRzoKIV55d1btVBa0spkMMoLqjHTIhVSimhTTwDOYeazo0LNTpHGWcHwnOwkktOrsV6ezhCt+TGLGyTlZ1bTmshEYPPM9O9TLJb9tvedWr4eP5LFflDGonhtnIjfupkHqFCy8EmVqGsHmMQR1jSQ6ybucFexq2akLPDHwk8mBecmMB4rNEo0woP2pIxNkYE1TqSIo+liRdWfHrqKtDACnVowDaI6Rcj+pPNt1YIo3Yz2nn4r61IMjMHpY+3+2Z1Vs/vVotNxb1UZs6/uPVBloX1XWr2SU+15iYr6pxmzp0CxdnmWvnxq9zzU8o11xiZpokQldPs8rAIpAtWjeD22713lLNb11Jwjm8fW0Bq6v5pHSn0peMH6mqa3m5a62o/Ve9rU/DE0yiunPOzr4H57jGzRFZGsOM11Qdeok13GsPk1XzldrXfJnW56qE36v5fsGP9f3/fKPq2LMbYw7pl3LkhnzFtpuibdG6pWh7HTx+VsGjMAtfcaVozHlhjNCTFBe6jnng2J7lDt7yk7PpaDY5p3KIeqP8IpVCi1DwAB7dPZq7er8Utra+e5l8cc4jqUuDnWP+y6UNqp1Nvy1qU69LmZ/KUmY1OLf+7zL5n8uzCzKcTYm7uxiwV+4dh4MHQh9JVGhrjuU1JoCFHRDGnegwANflQPJTI2jEW7MdQJ7nBTCgXCEBKI7hAOpy2J4DMktn80MKPviQ1K7chuqsnFD8Op67PUyJwbRhycG63H/RXTR/gUncrcgscpVdh+g5EeYG8ruSc21jcKXRoCc/xefmZwqPqFKnQs4Gq+W4ALqkkoqDe+2c+1eYZgF0O504L4jF1k7d3sF7zNru2xmq2tyDxtS93fcYUQs0lmvGLK2GOUOBAFTDmBmygy6q5nXwwyNSrVMY0+38+CL5Jd94NBsmQuLW313chiOqDSkFWoAbhtMRJtDPGLcNEykNT+gQlU+24dGIKVBZaoMzqBFyDkMu+q4+yZLhdZDIqWZjtNCpjNMkItuQ4NC1SrydShyw5xi5S+mX3vHh/YRPQCR2pWHJGA84S9An/r3jj461kEi24a6IY5HAk7vHEDGpiD9kum3/OvaJ3/+ObNu/xcBo2DZ/ip9qnLRLQiZTzFLbeqHIjq9OU7Lj9+kJ2fF1nJKd3yPb8IRKJjIFh/fuK+KnUjzDUBOfRUjbbp4Uz8jOH1wjn3t4/8699+77cbT1k6vbK3uJfht5DDY0EfLBiokfrt3p9Y45ysYsQgX0XP1chGxvw6EJt5wXzTI5h48EMPeg8s7c2s78rHVXxBMv/x0Q8vTp01AkSnAkX7anyYyM55nZt8vZoLRxzUZ4e/r0KSGPRrZxLqZJBO6FiFqpW5E0W7lyEYG5J0WXRCiSARtmbrVv5b9jW8EK8R8fNmSo93VZ7ra34bFdYPXx0MSt1YIr1FCrFN3WMsONNHJHqcy2uExE5roZnpbrnloT5/tZ0a7beSFNIBGn1W63x4fN5jPoT2ybmiG+WQtartSy46zkyGt2ghqdJWwZ1LLiUTtCjto1mFX0Ur4UWwixfFG5YB5FEmMxNo6SA7oKFRGnIkHb9qWUCBnVRZwqoe+QyO2Cigf4ZOeI/EbI2q5ul5vXG7vUziZ8ndnBNjuKjPWbJ5eN7Hs39m+Rnd+/SH7FnCVMohFZmeP5ZOvfLkSYokm0Q+aaANxJbsk3M4c8gpTZjmfFUL7SQtIh+kMhhhxpypSJL+59vsuguh3/ht8hERuiucyrEe3d3Av6nU6/F+3v4q3OAe0d0MHgAAf7+53Bjd5uuB/uRrf60W53d3ATKb1Jb/b3BhTD/b1btwY3DvZpf48MMTFOa8Rv9TrdW163593oPOruBb2bQXff392/2bmx99VON+h0WmTnTy80dGB1+d0X6KAuxM7LayUUScS06zyoOjBp/dkF8v0L5jCkvJK0uJRmlk65k8MNpp7iu6f9asbWBESZnq3sgJ5Lr+zyehLTIzv/9JvE8zyyDcc2mWnkfO2V15RP5BbSzGPnrhv1rl7bxrzG5eIR4xzlCy8T87uvuEbk2fULbg1NkovEXM8EzQ6vn3kD12fSNp/+dqqN0VPtrPp5NU59NrH06tqY6oQ3alWaF77SjpT/u85mp8KLj4MFxeQmG69R06iFlrf2BaXPg85caTO/8p+3eHmeI6nRiEbO2Yb2GgpLoVC2hvVepgNsnt+NgtmrafJaxMTaZtvIdJubb30Tbqi6erfUouLmImaWt70sZr1oeVlYAwoc9ysaWxoF5vVaNWbx5tU3Y9RIv+J2iwbtShW3/FTrueWnXtktP80ab/mpV3sblA6WErIV4PUicLWZ42X7NF6H4eVheL6DYfX1e6MGhvbGfQqLo8CCXoSDzsGld6/mWd3/BwAA//9H/OsIaz4AAA==
kind: ConfigMap
metadata:
  creationTimestamp: "2020-06-22T12:09:03Z"
  labels:
    MODIFIED_AT: "1592827743"
    NAME: kube-ops-view
    OWNER: TILLER
    STATUS: DEPLOYED
    VERSION: "3"
  name: kube-ops-view.v3
  namespace: aleol57
  resourceVersion: "555991060"
  selfLink: /api/v1/namespaces/aleol57/configmaps/kube-ops-view.v3
  uid: ba7497cf-5a60-48eb-9d56-0f6c0c176a8a
$ helm mapkubeapis --v2 --tiller-out-cluster --release-storage configmaps --namespace aleol57 kube-ops-view
2020/06/24 09:26:43 Release 'kube-ops-view' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2020/06/24 09:26:43 Get release 'kube-ops-view' latest version.
Error: Failed to get release 'kube-ops-view' latest version: release: "kube-ops-view" not found
Error: plugin "mapkubeapis" exited with error
$ helm plugin list | grep mapkubeapis                                                        
mapkubeapis     0.0.14  Map release deprecated Kubernetes APIs in-place

Is it safe to only update the api version, if the content is different for the new version

When moving from one api version to another, we sometimes need to change the content of the yaml definition of the object.
Is it safe from helm's perspective for the mapkubeapis plugin to only change the version and not the content?

This was mostly answered in #34 but in a helm dev call there was a mention that the three-way merge could be something to think about with respect to this issue.

For example, in kube 1.22, the Ingress resource versions extensions/v1beta1 and networking.k8s.io/v1beta1 are removed and we must migrate to networking.k8s.io/v1. However, when migrating to the new version, some fields must be changed and some added (https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122).

Is it safe for the mapkubeapis plugin to ignore those field changes? Could helm's three-way merge logic be affected by a helm manifest that differs from the actual yaml stored in kubernetes?

If there could be issues, we may want to look at using the kubectl convert plugin to apply all changes to the helm manifest.
In fact, using the plugin may be interesting regardless as it has the potential to handle every format properly.

@hickeyma I'm hoping you may have an opinion about this.

mapkubeapis looks on last deployment but helm looks on last non-failed deployment

I had today a situation where a helm deployment had such history:
Revision 1 - status deployed
Revision 2 - status failed
Revision 3 - status failed

I run helm mapkubeapis NAME_OF_RELEASE and it showed that there were no changes needed.
Then I run helm upgrade --install and it faield that PDB has version policy/v1beta1.

It seems that helm tries to upgrade from last successful revision, but helm-mapkubeapis takes into account only last one.

As a workaround I deleted form kubernetes secrets of helm releases that are failed and run helm mapkubeapis NAME_OF_RELEASE

Plugin version not updated for v0.4.0

The plugin.yaml still shows v0.3.2:

plugin.yaml 
name: "mapkubeapis"
version: "0.3.2"
usage: "Map release deprecated Kubernetes APIs in-place"
description: "Map release deprecated Kubernetes APIs in-place"
command: "$HELM_PLUGIN_DIR/bin/mapkubeapis"
hooks:
  install: "cd $HELM_PLUGIN_DIR; scripts/install_plugin.sh"
  update: "cd $HELM_PLUGIN_DIR; scripts/install_plugin.sh"

This causes a helm plugin ls to show a wrong version number.

this is a great helm plugin. thanks ~

we have a lot of deployment made by helm in k8s v1.14. we upgrade k8s to 1.18.

then The annoying thing is coming。offical helm v2 does not have the right way to slove this .
even I change the chart apiserver to apps/v1.

Error: UPGRADE FAILED: failed decoding reader into objects: unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta2"

you guys doing the great things .

Passing tls and tiller-namespace?

I'm trying to use 0.2.0 version of this plugin with Helm 2 version. Helm is configured with tls and tiller is deployed in separate namespace, therefore, I have to use --tls and --tiller-namespace flags.

Trying to run helm mapkubeapis prometheus-operator --namespace monitoring --dry-run --tls --tiler-namespace=tiller but it doesn't work (throwing error about this additional flags)

Cannot download helm-mapkubeapis_0.0.15_linux_amd64.tar.gz

Hello,

I tried installing the plugin but I get a gzip error when running the install command

$ helm plugin install https://github.com/hickeyma/helm-mapkubeapis
Downloading and installing helm-mapkubeapis v0.0.15 ...
https://github.com/hickeyma/helm-mapkubeapis/releases/download/v0.0.15/helm-mapkubeapis_0.0.15_linux_amd64.tar.gz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Error: plugin install hook for "mapkubeapis" exited with error

I tried the link (above) and noticed the tar file is inaccessible (404)
Where do I get the tar file to install the plugin?

Thanks

Allow removal of resources for APIs that do not exist anymore

I was experimenting with helm-mapkubeapis and noticed that it currently does not offer a way of removing resources. However, in the case of the PodSecurityPolicy API, there is no new version to apply. It would be useful to have logic to allow removals when applicable.

Getting issue to run helm mapkubeapis on GKE cluster no Auth Provider found for name "gcp"

Getting issue to run helm mapkubeapis on GKE cluster

helm mapkubeapis XXXX -n XXX 2023/03/15 18:36:58 Release 'XXXXX' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions. 2023/03/15 18:36:58 Get release 'XXXXX' latest version. Error: failed to get release 'XXXXX' latest version: query: failed to query with labels: no Auth Provider found for name "gcp" Error: plugin "mapkubeapis" exited with error

Error on new helm deployment

Firstly I love this plugin, it has saved us so much pain

One thing though, when we start a new service that has never been deployed to AKS, it generates and error. It doesn't really make sense for this to be an error situation though

image

"no deprecated or removed APIs" even thought there are some (Mapfile used)

Hey there,

I currently have a problem with this otherwise great plugin: I have a v2 Helm release on a cluster recently upgraded to k8s 1.17. helm upgrade fails with the known unable to recognize [...] error. I deleted all failed releases and am now in a state where I have a DEPLOYED helm as the latest, which has several removed/deprecated apiVersion entries in it. When I run mapkubeapis on this release, it tells me Release 'myrelease' has no deprecated or removed APIs.. But when I run kubectl get configmap -n mynamespace myrelease.v2 -o=jsonpath='{.data.release}' | base64 -d | gzip -d | grep v1beta1 against it, it results in the following output (note the whitespace):

apiVersion: extensions/v1beta1
apiVersion: apps/v1beta1
  apiVersion: extensions/v1beta1

I figured the corresponding resources were (shortened for readability):

apiVersion: extensions/v1beta1
kind: Ingress
[...]
apiVersion: apps/v1beta1
kind: StatefulSet
[...]
deployment:
  apiVersion: extensions/v1beta1
  kind: Deployment

So I thought (after reading #11 ) it may be caused by something regarding whitespace and line endings. I tried to duplicate the mappings like so:

  - deprecatedAPI: "apiVersion: extensions/v1beta1\nkind: Deployment"
    newAPI: "apiVersion: apps/v1\nkind: Deployment"
    deprecatedInVersion: "v1.9"
    removedInVersion: "v1.16"
  - deprecatedAPI: "apiVersion: extensions/v1beta1\n  kind: Deployment"
    newAPI: "apiVersion: apps/v1\nkind: Deployment"
    deprecatedInVersion: "v1.9"
    removedInVersion: "v1.16"

in order to eliminate the third case and also created duplicate mappings with \r\n in them to get rid of the other two. In the end, I had created 6 versions of each mapping (720 lines in total), but unfortunately, it didn't help and mapkubeapis still said it didn't find any deprecated/removed apiVersions.

This leads me to the following two questions:

  • What am I doing wrong / Why doesn't it detect them?
  • Would it be possible to implement reqex replacement (instead of plain strings) so that we can avoid duplicate entries in the mappings file?

Support for non-Kubernetes APIs

Today, I was facing an error message after upgrading the Kubernetes cert-manager from version 1.5.4 to 1.6.1 when deploying an application update via Helm:

Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "Certificate" in version "cert-manager.io/v1alpha3"
helm.go:88: [debug] unable to recognize "": no matches for kind "Certificate" in version "cert-manager.io/v1alpha3"
current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes

Okay, cert-manager removed the alpha and beta APIs, so I expected their absence. The Helm docs recommend updating charts before actually removing APIs, but that sometimes is out of my control. Also, it can put me in a kind of chicken-and-egg situation where I need to have a dependency already present in the cluster before I can update a Helm release.

Luckily, helm-mapkubeapis to the rescue! I have added

  - deprecatedAPI: "apiVersion: cert-manager.io/v1alpha1\nkind: Certificate"
    newApi: "apiVersion: cert-manager.io/v1\nkind: Certificate"
    deprecatedInVersion: "v1.19"
    removedInVersion: "v1.22"
  - deprecatedAPI: "apiVersion: cert-manager.io/v1alpha2\nkind: Certificate"
    newApi: "apiVersion: cert-manager.io/v1\nkind: Certificate"
    deprecatedInVersion: "v1.19"
    removedInVersion: "v1.22"
  - deprecatedAPI: "apiVersion: cert-manager.io/v1alpha3\nkind: Certificate"
    newApi: "apiVersion: cert-manager.io/v1\nkind: Certificate"
    deprecatedInVersion: "v1.19"
    removedInVersion: "v1.22"
  - deprecatedAPI: "apiVersion: cert-manager.io/v1beta1\nkind: Certificate"
    newApi: "apiVersion: cert-manager.io/v1\nkind: Certificate"
    deprecatedInVersion: "v1.19"
    removedInVersion: "v1.22"

to a Mapfile and ran helm mapkubeapis, which worked as expected (or hoped?). So, the actual issue is technically solved for me.

The only thing left (and reason to raise this issue) is that helm-mapkubeapis refers to Kubernetes versions in deprecatedInVersion and removedInVersion while the API discussed here is a CRD served by the Kubernetes cert-manager. So, I wonder if there is a sound way to track API versions and deprecations from 3rd party providers and their versioning instead of Kubernetes.

Offline installation of helm plugin.

Hey,

Is there a way we can do offline installation of the helm-mapkubeapis plugin without hitting internet.
I did try to modify the install_plugin.sh script with my internal repo and pushed helm-mapkubeapis_0.3.0_linux_amd64.tar.gz to my local repo but the installation is not successful.

Question: Removal of Helm v2 support?

Helm v2.17.0 was the final release of Helm v2 in October 2020. Helm v2 is unsupported since November 2020, as detailed in Helm 2 and the Charts Project Are Now Unsupported.

Helm v2 uses an older version of Kubernetes Go client which is incompatible with Kubernetes Go Client 1.18+. helm-mapkubeapis calls Helm v2 storage APIs which in turn call Kubernetes Go client APIs. This means that the plugin is restricted to using Helm v3.1.3 (last Helm v3 release which uses Kubernetes Go Client pre 1.18) which was released in April 2020.

Is it time to drop support for Helm v2 in this plugin as Helm v2 is no longer supported? The benefit is that we can update to the latest dependencies.

hpa in 1.26 k8s is not mapped properly

Scenario

Error: UPGRADE FAILED: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "platform-identity-keycloakx" namespace: "test" from "": no matches for kind "HorizontalPodAutoscaler" in version "autoscaling/v2beta2"
ensure CRDs are installed first
helm.go:84: [debug] resource mapping not found for name: "RELEASE_NAME-keycloakx" namespace: "default" from "": no matches for kind "HorizontalPodAutoscaler" in version "autoscaling/v2beta2"
  • helm mapkubeapis RELEASE_NAME produced
2023/05/04 11:42:07 Release 'RELEASE_NAME' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2023/05/04 11:42:07 Get release 'RELEASE_NAME' latest version.
2023/05/04 11:42:09 Check release 'RELEASE_NAME' for deprecated or removed APIs...
2023/05/04 11:42:09 Finished checking release 'RELEASE_NAME' for deprecated or removed APIs.
2023/05/04 11:42:09 Release 'RELEASE_NAME' has no deprecated or removed APIs.
2023/05/04 11:42:09 Map of release 'RELEASE_NAME' deprecated or removed APIs to supported versions, completed successfully.

Problem

Cannot upgrade or uninstall the chart:

uninstall.go:117: [debug] uninstall: Failed to delete release: [unable to build kubernetes objects for delete: resource mapping not found for name: "RELEASE_NAME-keycloakx" namespace: "default" from "": no matches for kind "HorizontalPodAutoscaler" in version "autoscaling/v2beta2"

Helm manifest still has deprecated apiVersion:

helm get manifest RELEASE_NAME
...
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: RELEASE_NAME-keycloakx
...

map file doesn't match \r\n

i suspect helm charts created with macosx have an issue where my ingress was not getting matched because the file had \r\n instead of \n like in the map file.
my workaround was to use the following mapping

  • deprecatedAPI: "apiVersion: networking.k8s.io/v1beta1\r\nkind: Ingress\r\n"
    newAPI: "apiVersion: networking.k8s.io/v1\nkind: Ingress\n"

but i'm sure a quick string replace would be a better solution for this.

selector not generated for Deployment when upgrade extensions/v1beta1 to apps/v1

background

we have a old version(extensions/v1beta1) of configuration for Deployment that didn't set selector field, this has been running in k8s 1.14. While migrating to k8s 1.16, we found extensions/v1beta1 has deprecated. for the new version 'apps/v1', the selector field is now a required field according to schema. than i use this plugin to upgrade configuration in cluster

issue

although we have the apiVersion upgraded, but the selector field is not generated, and cause some issues to do further helm upgrade

Any thoughts, ideas will be appreciated!

help should be shown if no arguments are provided

$ helm
The Kubernetes package manager

Common actions for Helm:

- helm search:    search for charts
- helm pull:      download a chart to your local directory to view
- helm install:   upload the chart to Kubernetes
- helm list:      list releases of charts
...

Actual results:

$ helm mapkubeapis
Error: name of release to be mapped needs to be passed
Error: plugin "mapkubeapis" exited with error

Expected results:

$ helm mapkubeapis
Map release deprecated or removed Kubernetes APIs in-place

Usage:
  mapkubeapis [flags] RELEASE

Flags:
      --dry-run                  simulate a command
  -h, --help                     help for mapkubeapis
      --kube-context string      name of the kubeconfig context to use
      --kubeconfig string        path to the kubeconfig file
      --mapfile string           path to the API mapping file (default "~/helm/plugins/helm-mapkubeapis/config/Map.yaml")
      --namespace string         namespace scope of the release. For Helm v2, this is the Tiller namespace e.g. kube-system
  -s, --release-storage string   for Helm v2 only - release storage type/object. It can be 'secrets' or 'configmaps'. This is only used with the 'tiller-out-cluster' flag (default "secrets")
      --tiller-out-cluster       for Helm v2 only - when Tiller is not running in the cluster e.g. Tillerless
      --v2                       run for Helm v2 release. The default is Helm v3.

Patch fails if release has a comment in templates

helm-mapkubeapis failed to patch a release because in its manifest it had a comment between apiVersion and kind:

...
apiVersion: extensions/v1beta1 # Daemonset is still not part of v1 api version\nkind: DaemonSet
...

Still a speculation, not 100% sure that it was the comment but this release was the only one that wasn't mapped (out of hundreds we had).

mapkubeapis only updates last release even if it is in FAILED state

We recently had an issue with mapkubeapis only updating last release even if it is in FAILED state.
The cluster was updated, there was a pipeline deployment that failed with the correct apis but as mapkubeapis updated only the last failed we could not get the build to success.
To fix it we delete all failed releases (but not the deployed one) replayed mapkubeapis that then updated the deployed one and after that we were able to push the pipeline since the previous version was updated.

Sugestion by default war if last release is failed and ask to remove all failed ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.