yonahd / kor Goto Github PK

some refs:

• cli/cli@d9ef40c
• cli/cli#900

Describe the bug
Helm creates unused Secrets https://github.com/yonahd/kor/blob/main/charts/kor/templates/secret.yaml
when slack token is not given

To Reproduce
Run Helm install without providing slack tokens

OS version, architecture and kor version
e.g. Darwin x86_64
Kor version: 0.2.4

Find Kubernetes Ingresses that are not used. -> Not pointing to any services
The command should be ing or ingress

Describe the bug
retrieveConfigMapNames results in panic

To Reproduce
upgraded to EKS 1.28, tried to reinstall the helm to ensure i have latest image

Expected behavior
no panic

OS version, architecture and kor version
Arch: amd64
Kor version: v0.2.7 (I hope, would be great to print version during startup)

Additional context

Server listening on :8080
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x15ec483]

goroutine 12 [running]:
github.com/yonahd/kor/pkg/kor.retrieveConfigMapNames({0x1c37550?, 0xc00034d040?}, {0xc000392340, 0x10}, 0x0)
        /build/pkg/kor/confimgmaps.go:95 +0x1a3
github.com/yonahd/kor/pkg/kor.processNamespaceCM({0x1c37550, 0xc00034d040}, {0xc000392340, 0x10}, 0x0?)
        /build/pkg/kor/confimgmaps.go:126 +0x25f
github.com/yonahd/kor/pkg/kor.getUnusedCMs({0x1c37550?, 0xc00034d040?}, {0xc000392340, 0x10}, 0x1c37550?)
        /build/pkg/kor/all.go:23 +0x5d
github.com/yonahd/kor/pkg/kor.GetUnusedAll({{0x0?, 0x0?}, {0x0?, 0x0?}}, 0x0?, {0x1c37550, 0xc00034d040}, {0x19c3d21, 0x4}, {0x0, ...})
        /build/pkg/kor/all.go:129 +0x12d
github.com/yonahd/kor/pkg/kor.exportMetrics({{0x0?, 0xc000390540?}, {0x0?, 0xc0000607d0?}}, {0x1c37550, 0xc00034d040}, {0x19c3d21, 0x4}, {0x0, 0x0, ...})
        /build/pkg/kor/exporter.go:52 +0x185
created by github.com/yonahd/kor/pkg/kor.Exporter
        /build/pkg/kor/exporter.go:34 +0x235

When a user gets prompted to delete a resource and chooses not to delete, offer to label the resource as used using this label feature https://github.com/yonahd/kor#ignore-resources

In the output options support outputting in yaml format
Using https://github.com/kubernetes-sigs/yaml/blob/v1.3.0/yaml.go#L82 we can support yaml and json using the same function.

Describe the bug
When running kor with output yaml or json, parsing with jq or other tools fails because of the kor banner.

To Reproduce
Kor cm - - output json

Expected behavior
Skip banner printing if not table output.

OS version, architecture and kor version
e.g. Darwin x86_64
Kor version: 0.2.7

I think we should merge GetUnusedDeployments(includeExcludeLists IncludeExcludeLists, kubeconfig string) and GetUnusedDeploymentsJSON(includeExcludeLists IncludeExcludeLists, kubeconfig string) (string, error) into something like GetUnusedDeployments(includeExcludeLists IncludeExcludeLists, kubeconfig string, format string).

Because they have many duplicated codes and provide similar functionality, the only difference is the output format. I see another issue is talking about adding yaml output format. At that time we may have 3 similar functions. The case also apply on other resources function GetUnusedXXX.

Refactor the Helm chart to allow adding the slack information such as the api token from a secret.

Is your feature request related to a problem? Please describe.
Currently, when someone creates a POC or test deployment they need to remember to delete it when they are done.
This feature would allow users to mark these in advance so KOR can find them later and remind us to delete them

Describe the solution you'd like
All resources with the label
kor/used = false
would be marked as unused

Additional context
The project looks for the reverse of the label already

Find namespaces with no resources

Add tests for the Kubernetes unused resource finders.
In the first step use "k8s.io/client-go/kubernetes/fake"

Using a multistage Dockerfile create a Docker image that can run kor.
The final image should be based on alpine

I'm a new user of Kor, and I'm already impressed with how great of a tool it is! With that said,
I'd like to propose an enhancement or new feature for consideration.

Enhancement Proposal

addition of a --delete option for unused resources in the kor tool.
This enhancement would provide users with the ability to delete resources that are identified as unused.

Expected behavior

The deletion process should be initiated for the unused resources.
The command should provide feedback on the deletion process, indicating which resources are being deleted and their status.

Is your feature request related to a problem? Please describe.
When running in cluster we should be able to clearly see the version of kor on startup

Describe the solution you'd like
Print the version on startup
e.g. v0.2.7

As the CLI is supporting outputing unused resources which will lead probably to deletions for the cluster, i would like to have --delete flag which executes deletions on the output of the command, Example:

kor secrets -n default --delete

secret foo deleted

Also important to support deletions from all namespaces, will be useful for Cronjobs across the cluster

Describe the bug
KOR is reporting a kubernetes secret as unused but it is getting used by one of deployment in init containers like below:

initContainers:
 - command:
      - alembic
      - upgrade
      - head
      env:
      - name: MY_TEST_PASSWORD
        valueFrom:
           secretKeyRef:
              key: password
              name: test-first-admin

KOR-OUTPUT

kor secret -n test

Unused Secrets in Namespace: test
+---+----------------+
| # | RESOURCE NAME  |
+---+----------------+
| 1 | test-first-admin |
+---+----------------+

Expected behavior
This should not report "test-first-admin" secret as unused secret in test ns.

OS version, architecture and kor version
e.g. Ubuntu x86_64
Kor version: 0.1.9

First of all I would like to express my great interest in this project.

And I clone this project to use ( casue of have no arm64/darwin yet), but I noticed that there might be some problems.
KOR could not find resources used by crd.

For example, I use traefik ingressroute for my cluster ingress, and I would use cert signed by cert-manager, which store in kubernetes secret, but KOR treats it as orphaned resources.

I don't have a particularly good solution at the moment, so I hope to discuss it in this issue. Adding all CRDs is obviously unrealistic

Add a flag --exclude which will make kor run on all namespaces except the one included

Feature request from my work with the tool, I would like in 1 command to get the output from all namespaces in Table / JSON format to be able to iterate on all namespaces to perform deletions.
Instead of today when there is a separate Table for each namespace i would like to get 1 table which have Namespace, resource type, and resource name in columns.

Allow the output to be in multiple forms within the format.
e.g.

• Namespace
• Resource
• All

Summary
I would like to suggest that the Kor project be updated to support Prometheus metrics.
CLI is very useful but allows users to collect and monitor Kor's metrics using Prometheus giving consistent visibility to the Kubernetes Orphaned Resources on your cluster.

The repo is yonahdissen/kor

Describe the bug
I've deployed kor using helm provided in README and it worked in our test cluster (EKS 1.27), however it crashlooped on this error in our "legacy" EKS cluster (which runs 1.27 but is here with us for quite a while) on this error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x15d54f3]

goroutine 23 [running]:
github.com/yonahd/kor/pkg/kor.retrieveUsedIngress({0x1c158d0, 0xc0002c5380}, {0xc000046e50, 0xa})
        /build/pkg/kor/ingresses.go:48 +0x4b3
github.com/yonahd/kor/pkg/kor.processNamespaceIngresses({0x1c158d0, 0xc0002c5380}, {0xc000046e50, 0xa})
        /build/pkg/kor/ingresses.go:78 +0x4b
github.com/yonahd/kor/pkg/kor.getUnusedIngresses({0x1c158d0?, 0xc0002c5380?}, {0xc000046e50, 0xa})
        /build/pkg/kor/all.go:105 +0x5d
github.com/yonahd/kor/pkg/kor.GetUnusedAllStructured({{0x0?, 0x0?}, {0x0?, 0x0?}}, {0x1c158d0, 0xc0002c5380}, {0x19a42d6, 0x4})
        /build/pkg/kor/all.go:204 +0xa1f
github.com/yonahd/kor/pkg/kor.exportMetrics({{0x0?, 0xc000131560?}, {0x0?, 0xc00005a7d0?}}, {0x1c158d0, 0xc0002c5380}, {0x19a42d6, 0x4})
        /build/pkg/kor/exporter.go:52 +0x145
created by github.com/yonahd/kor/pkg/kor.Exporter
        /build/pkg/kor/exporter.go:34 +0x190

To Reproduce

We use https://github.com/nginxinc/kubernetes-ingress in production and we're trying ALB controller as ingress on test cluster. I've listed on ingresses in all namespaces and found it that some have class <none>. This may be helpful.

Installed today (2023-10-19) using:

helm upgrade -i kor \
    --namespace kor \
    --create-namespace \
    ./charts/kor

Expected behavior

No panic.

Screenshots

OS version, architecture and kor version
EKS 1.27 AWS managed AMI (Amazon Linux), kernel 5.10, amd64
Kor version: latest images as of 2023-10-19

Additional context

I just tested the latest version, and running "kor all" aggregates the results, i.e., the list is not cleared before the next namespace is checked, so the previous results get merged in the next namespace's results.

Describe the bug
kor cli doesn't output in JSON format while in table mode it is generating output normally

To Reproduce
Running:
kor all -n default --output json

Expected behavior
Output in JSON format

Screenshots
If applicable, add screenshots to help explain your problem.

Mac os M1
Kor version: 0.1.9

In order to allow user to run the tool on a specific namespace
Create -n flag which would apply to all resources that can be namespaced.

e.g. kor cm,secret -n my-namespce

Is your feature request related to a problem? Please describe.
While Kor is an excellent tool for identifying unused Kubernetes resources, it currently does not support identifying unused Custom Resource Definitions (CRDs). In large Kubernetes clusters, it's not uncommon to have CRDs that were once used but no longer have any instances created from them. Identifying these unused CRDs can help in cleaning up and maintaining a lean cluster configuration.

Describe the solution you'd like
I'd like Kor to have the capability to identify and list unused CRDs, i.e., CRDs for which no instances have been created. This can be an additional subcommand, similar to the existing ones like configmap, secret, etc. For example: kor crd could list all the unused CRDs in the specified namespace or all namespaces.

Describe alternatives you've considered
One could manually check for CRD instances using kubectl, but this is cumbersome and not scalable for large clusters. Having this feature integrated into Kor would streamline the process and make it consistent with the identification of other unused resources.

Additional context
As Kubernetes continues to evolve and organizations adopt more custom resources, the ability to identify unused CRDs will become increasingly important for cluster hygiene. Adding this feature would further enhance Kor's value proposition as a comprehensive Kubernetes resource management tool.

Hi! Cool project - came across it from natan's post!

This is less of an issue and more of a discussion, but i'd love to have a way to ignore helm secrets from the output.
As you know - helm v3 uses secrets as its release storage layer and i am not sure i would consider those are orphan or unused. perhaps it seem that way but once you will want to perform some helm-related actions, you may need those.

What do you think about that? I don't mind opening a PR once we agree on a small design. e.g we could make it not specific to helm, something like an --exclude option, with a pattern (regex?) being passed that will be used as a filter applied on the resource name.

in my case it would be something like

go run main.go secret --namespace default --exclude 'sh\.helm\.release\.v1\.([a-zA-Z0-9-]+)\.v(\d+)'

Shay.

I was thinking of creating a project using kor as a go library, the goal is an agent running inside the kubernetes cluster and periodically logging unused resources and exporting metrics, for this to be possible the conventional method is to create a clientset using the service account that the pod uses.
(rest.InClusterConfig() uses the mounted SA inside the pod)

currently kor passes kubeconfig to the functions instead of the clientset (that could be created from an SA) so you can't use the kor library without a kubeconfig inside of the pod.

Would it be useful if we implement the check to list pods that are not associated with any deployments ? I am not sure if we consider these pods as "not used" since it is perfectly possible to create such pods, however it might also be useful if we can list them.

Describe the bug
different styles of naming resource types and commands.
some resources accept only single or plural form and some only have the short name.

for example valid subcommands are:
ingress - single
services - plural
pdb - shortname
while they have inconnsistant invalid forms:
ingresses - plural is invalid
service - single is invalid
poddistruptionbudget/s - both single and plural are invalid

Also nothing is printed when an unsupported subcommand is found making it even more confusing.

To Reproduce
Steps to reproduce the behavior:
try to use the kor cli with commands like service vs services.
kor service vs kor services - only plural is valid
kor secrets vs kor secret - only single form is valid

Expected behavior
kor should accept all 3 forms: single, plural and shortnames for supported resources based on what kubectl allows and the api endpoint.

kor should also tell you through rootcmd when it doesn't find a supported resource type.
for example: kor pv should output something like resource type "pv" is not supported use 'kor --help' for available subcommands

Screenshots

OS version, architecture and kor version
any OS and architecture, tested on windows.
Kor version: v0.2.1

Additional context
The reason for this discrapency in form is usually because when using kubectl users usually use the single form (kubectl get deployment) while the api names are in plural form (GET /apis/apps/v1/deployments) . in kubectl using deploy, deployment and deployments are all valid options.

Is your feature request related to a problem? Please describe.
I am using Kor to scan (and delete in the future) unused resources in my cluster, but I have to run it manually every time I want to clean up my cluster. This is tedious and time-consuming, and sometimes I forget to do it.
I would like to have a way to automate this process and run it periodically without my intervention.

Describe the solution you'd like
I would like to have a flag or option in Kor that allows me to schedule the scan and deletion of unused resources at a regular interval, such as daily, weekly, or monthly. This would automate the process of cleaning up my cluster and keep it optimized and efficient. For example, I could run kor -interval daily to run the scan and deletion every day at a fixed time.

Describe alternatives you've considered
I have considered using cron jobs in Linux or Github actions to run Kor at a regular interval, but they are not very convenient or flexible for me. I have to create and maintain a separate script or command for each cluster or namespace that I want to scan and delete, and I have to deal with different syntaxes and configurations for each platform.

Additional context
Add any other context or screenshots about the feature request here.

Do something similar to https://github.com/ibuildthecloud/finalizers

Kor should be able to find unused pdbs.
Add the command and logic.

Describe the bug
When running kor with --output json we get different responses from resources
e.g.

{
  "default": {
    "ConfigMap": [],
    "Deployment": null,
    "Hpa": null,
    "Ingress": [],
    "Pvc": [],
    "Role": [],
    "Secret": [],
    "Service": null,
    "ServiceAccount": [],
    "Statefulset": null
  }
}

Expected behavior
All resources should return []

OS version, architecture and kor version
e.g. Darwin x86_64
Kor version: 0.1.9

Describe the bug

All "standard" kubectl-like clients respect the KUBECONFIG environment variable, but kor does not use it, defaulting to ~/.kube/config.

To Reproduce

Steps to reproduce the behavior:

$ rm ~/.kube/config  # :trollface: - it's my default since I think having multiple dev+prod contexts in one central file is dangerous

$ export KUBECONFIG="${HOME}/.kube/config-some-other-file"

$ kor secret -n kube-system
Failed to load kubeconfig: stat /Users/myname/.kube/config: no such file or directory

Expected behavior

Respect the environment variable and don't explicitly require --kubeconfig "$KUBECONFIG". If --kubeconfig is set though, that argument should have higher priority (as with other clients).

OS version, architecture and kor version

macOS Ventura 13.5, arm64
Kor version: 0.1.8

Kor should be able to find hpa's that are not pointing at any service. Add the command and logic.

Example use case:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test
  namespace: test
rules: null

Is your feature request related to a problem? Please describe.
I would like to have more control over the criteria and scope of the search for unused resources. For example, I may want to exclude some resources that have certain labels or annotations, or only list resources that are older or larger than a certain threshold.

Describe the solution you'd like
I would like to have more flags or options to customize and filter the search query. For example, I could use something like --exclude-label owner:alice or --min-age 24h or --min-size 1Mi to refine the results.

Describe alternatives you've considered
An alternative solution could be to use another tool or script to filter the output of yonahd/kor, but that would be less convenient and efficient.

Additional context

When kor runs inside a kubernetes cluster it should mount a service account with Rbac instead of using a kubeconfig.

Make a inCluster flag

Label should be

kor/used=true

When the label is set to true kor should consider the resource as used.

Describe the bug
Unable to run kor because of redefined flags.

To Reproduce
just try to run kor on any system after commit 5a55f44 #120

PS F:\projects\kor> go run .

  _  _____  ____
 | |/ / _ \|  _ \
 | ' / | | | |_) |
 | . \ |_| |  _ <
 |_|\_\___/|_| \_\

panic: kor flag redefined: slack-webhook-url

goroutine 1 [running]:
github.com/spf13/pflag.(*FlagSet).AddFlag(0xc0000e2e00, 0xc0002fefa0)
        C:/Users/User/go/pkg/mod/github.com/spf13/[email protected]/flag.go:848 +0x606
github.com/spf13/pflag.(*FlagSet).VarPF(0x28?, {0x1e12ca0, 0x2a8eec8}, {0x1bbefa6, 0x11}, {0x0, 0x0}, {0x1be2068, 0x2a})
        C:/Users/User/go/pkg/mod/github.com/spf13/[email protected]/flag.go:831 +0x151
github.com/spf13/pflag.(*FlagSet).VarP(...)
        C:/Users/User/go/pkg/mod/github.com/spf13/[email protected]/flag.go:837
github.com/spf13/pflag.(*FlagSet).StringVar(0x1c30923?, 0x9b?, {0x1bbefa6?, 0x2a903e0?}, {0x0?, 0xc000068000?}, {0x1be2068?, 0x1c1cdf6?})
        C:/Users/User/go/pkg/mod/github.com/spf13/[email protected]/string.go:37 +0x69
github.com/yonahd/kor/cmd/kor.Execute()
        F:/devops projects/kor/cmd/kor/root.go:58 +0x288
main.main()
        F:/devops projects/kor/main.go:6 +0x17
exit status 2

Expected behavior
kor should return the output for root.cmd

OS version, architecture and kor version
Windows 10
Kor version: 0.2.6 after commit 5a55f44 #120

Additional context
Simply removing the duplicated flag on root.cmd lines 58-60 would solve the issue:
https://github.com/yonahd/kor/blob/5a55f449b86d91a62d7d80d7be743456698adeff/cmd/kor/root.go#L58C2-L58C2