[bug]: run k8sgpt analyze will only complete successfully once , after trivy integration is active about k8sgpt HOT 5 OPEN

Concurrent map writes indicate that this is a synchronization problem. I'll take a look. Thanks for reporting @liyuerich

from k8sgpt.

Small update, I tried to reproduce the issue with the the steps mentioned above. Unfortunately, I couldn't replicate this issue on Ubuntu 23.10, K8SGPT version: master

from k8sgpt.

I'm also getting intermittent "concurrent map writes" on 0.3.29 but I don't have trivy integration enabled, this seems to happen when the system is under load but even then I can't reproduce it reliably, just rerunning the command usually produces expected output.

I had it twice, both times I was running cluster-wide analysis (not limiting by the namespaces, having all filters enabled including Log with slows down the analysis significantly), kubeapi was also quite busy with other queries (first time installing several operators in parallel, second time running another scanning tool querying kubeapi as well), both times rerunning exactly the same command right after the failure succeeds.

k8sgpt version: 0.3.29
k8s version: v1.28.7 installed via brew
running on: Ubuntu 22.04.4 LTS, kernel 5.14.0-1054-oem,

CLI commands and output (first time it failed, didn't save the log from the second one):
k8sgpt_analyze_concurrent_map_writes.log

from k8sgpt.

It happened again today working with another cluster (same k8sgpt cli installation), working fine I removed log filter and run analyze again

#k8sgpt filters remove Log
Filter(s) Log removed
#k8sgpt analyze
fatal error: concurrent map writes
fatal error: concurrent map writes

goroutine 32 [running]:
k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypeWithName(0xc000268d20, {{0x40979af, 0x19}, {0x40590bf, 0x2}, {0x3770c97, 0x7}}, {0x4a541d8, 0xc00094e680})
k8s.io/[email protected]/pkg/runtime/scheme.go:174 +0x270
k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypes(0xc000268d20, {{0x40979af?, 0x0?}, {0x40590bf?, 0x0?}}, {0xc000bfa7f8?, 0x6?, 0xc0002bd4d0?})
k8s.io/[email protected]/pkg/runtime/scheme.go:148 +0x165
sigs.k8s.io/gateway-api/apis/v1.addKnownTypes(0xc000268d20)
sigs.k8s.io/[email protected]/apis/v1/zz_generated.register.go:60 +0x186
k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(...)
k8s.io/[email protected]/pkg/runtime/scheme_builder.go:29
github.com/k8sgpt-ai/k8sgpt/pkg/analyzer.GatewayClassAnalyzer.Analyze({}, {0xc000938ea0, {0x4a6c5b8, 0x65a8dc0}, {0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...})
github.com/k8sgpt-ai/k8sgpt/pkg/analyzer/gatewayclass.go:38 +0x11f
github.com/k8sgpt-ai/k8sgpt/pkg/analysis.(*Analysis).RunAnalysis.func3({0x4a3a9c0?, 0x65a8dc0?}, {0xc000904380, 0xc})
github.com/k8sgpt-ai/k8sgpt/pkg/analysis/analysis.go:268 +0xd9
created by github.com/k8sgpt-ai/k8sgpt/pkg/analysis.(*Analysis).RunAnalysis in goroutine 1
github.com/k8sgpt-ai/k8sgpt/pkg/analysis/analysis.go:266 +0x65e

Now k8sgpt analyze is failing even if I enable back Log filter, so 100% reproducible but I still have no idea how to trigger that on purpose, because I've enabled/disabled Log filter before without issue. I'm going to wait until tomorrow and see if reinstalling k8sgpt will fix it (I'll need it for a demo tomorrow).

from k8sgpt.

Hi, I'm adding support for external-secrets via integrations and see this issue when running go run . analyze. However, if I run go run . analyze --filter SecretStore I get the expected output.

go run . integrations list
Active:
> externalsecrets
Unused:
> trivy
> prometheus
> aws

 go run . filters list
Active:
> ClusterExternalSecret (integration)
> ClusterSecretStore (integration)
> Deployment
> Ingress
> SecretStore (integration)
> MutatingWebhookConfiguration
> ExternalSecrets
> Node
> Pod
> StatefulSet
> ValidatingWebhookConfiguration
> PersistentVolumeClaim
> ExternalSecret (integration)
> ReplicaSet
> PushSecret (integration)
> HorizontalPodAutoScaler
> Service
> CronJob
Unused:
> GatewayClass
> Gateway
> HTTPRoute
> PodDisruptionBudget
> NetworkPolicy
> Log

I attributed it to the number of analyzers I introduced. Each of those required an AddToScheme.

	err := v1alpha1.AddToScheme(client.Scheme())
	if err != nil {
		return nil, err
	}

Things seem to get better when I switched to using the following:

	var mutex = &sync.RWMutex{}

	mutex.Lock()
	err := v1alpha1.AddToScheme(client.Scheme())
	if err != nil {
		return nil, err
	}
	mutex.Unlock()

Seeing the reference to Trivy it made we wonder if the issue related to the way integrations loads an integration and executes it.

OS Dawrin 13.6.6
Branch: main
Kind cluster: v1.29.2

connecurrent-map-writes.txt

from k8sgpt.