The baaah from acorn-io

Add a descriptive README

Now that we're getting some differing contributors looking at this code a README explaining why this repo exists, how it can be used, etc. would be great.

#56 introduced dependency backoffs

likely that the dependencies aren't ready and the AppInstance is being re-enqueued. The logic he added backed off whenever the object was re-enqueued, not just on errors.
@thedadams also has context on this issue

Affected test on acorn-io/acorn: depends_test.go
Controller logs:

msg="Triggering [acorn-test-bkt5c/blue-night] [internal.acorn.io/v1, Kind=AppInstance] from [blue-night-e10a8366-97a/two] [policy/v1, Kind=PodDisruptionBudget]"
time="2023-03-13T20:02:56Z" level=warning msg="Backing off 5m42.138094065s for key acorn-test-bkt5c/blue-night on GVK internal.acorn.io/v1, Kind=AppInstance"
time="2023-03-13T20:02:56Z" level=info msg="Triggering [blue-night-e10a8366-97a/two-6c69d66b67-wj249] [/v1, Kind=Pod] from [blue-night-e10a8366-97a/two-6c69d66b67] [apps/v1, Kind=ReplicaSet]"
time="2023-03-13T20:02:56Z" level=info msg="Handling trigger [blue-night-e10a8366-97a/two-6c69d66b67-wj249] [/v1, Kind=Pod]"
time="2023-03-13T20:02:56Z" level=info msg="Handling [blue-night-e10a8366-97a/two] [apps/v1, Kind=Deployment]"
time="2023-03-13T20:02:56Z" level=info msg="Triggering [acorn-test-bkt5c/blue-night] [internal.acorn.io/v1, Kind=AppInstance] from [blue-night-e10a8366-97a/two] [apps/v1, Kind=Deployment]"
time="2023-03-13T20:02:56Z" level=warning msg="Backing off 5m57.129845482s for key acorn-test-bkt5c/blue-night on GVK internal.acorn.io/v1, Kind=AppInstance

reflect: call of reflect.Value.Interface on zero Value

i tried the traefik publisher (https://github.com/acorn-io/traefik-hub-publisher) and play around with it a bit by changing to watching corev1.Secret, then i got error

reflect: call of reflect.Value.Interface on zero Value)

router.Type(&corev1.Secret{}).Selector(sel).HandlerFunc(MyDummyHandler)

func MyDummyHandler(req router.Request, resp router.Response) error {
	return nil
}

i tried with other kind like Deployment or Ingress and it works fine.

the stacktrace is below

2022/06/29 17:49:19 INFO: Starting /v1, Kind=Secret controller
INFO[0000] Handling kube-system/bootstrap-token-abcdef /v1, Kind=Secret 
E0629 17:49:19.458016   33390 runtime.go:79] Observed a panic: &reflect.ValueError{Method:"reflect.Value.Interface", Kind:0x0} (reflect: call of reflect.Value.Interface on zero Value)
goroutine 373 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x101cfd4e0?, 0x140000a9950})
        /Users/anhtran/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:75 +0x7c
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0x140003ae120?})
        /Users/anhtran/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:49 +0x7c
panic({0x101cfd4e0, 0x140000a9950})
        /usr/local/go/src/runtime/panic.go:838 +0x204
reflect.valueInterface({0x0?, 0x0?, 0x1400083b468?}, 0x35?)
        /usr/local/go/src/reflect/value.go:1435 +0x100
reflect.Value.Interface(...)
        /usr/local/go/src/reflect/value.go:1430
github.com/acorn-io/baaah/pkg/router.statusField({0x101e9fe88?, 0x140006ce000?})
        /Users/anhtran/go/pkg/mod/github.com/acorn-io/[email protected]/pkg/router/save.go:49 +0xf8
github.com/acorn-io/baaah/pkg/router.statusChanged({0x101e9fe88?, 0x140006ce000?}, {0x101e9fe88, 0x1400013eb40})
        /Users/anhtran/go/pkg/mod/github.com/acorn-io/[email protected]/pkg/router/save.go:53 +0x30
github.com/acorn-io/baaah/pkg/router.(*save).save(0x140005e67b0, {0x101e9fe88, 0x140006ce000}, {{0x101eb0dc8, 0x1400070e000}, {0x101eb45a0, 0x1400013eb40}, {0x101eae1d0, 0x1400070db40}, {{0x0, ...}, ...}, ...}, ...)
        /Users/anhtran/go/pkg/mod/github.com/acorn-io/[email protected]/pkg/router/save.go:40 +0x144
github.com/acorn-io/baaah/pkg/router.(*HandlerSet).handle(0x140005e6700, {{0x0, 0x0}, {0x101a0ae75, 0x2}, {0x101c257be, 0x6}}, {0x140003c0150?, 0x1400084aa38?}, {0x101e9fe88, ...}, ...)
        /Users/anhtran/go/pkg/mod/github.com/acorn-io/[email protected]/pkg/router/handler.go:229 +0x594
github.com/acorn-io/baaah/pkg/router.(*HandlerSet).onChange(0x140005e6700, {{0x0, 0x0}, {0x101a0ae75, 0x2}, {0x101c257be, 0x6}}, {0x140003c0150, 0x22}, {0x101e9fe88, ...})
        /Users/anhtran/go/pkg/mod/github.com/acorn-io/[email protected]/pkg/router/handler.go:199 +0x268
github.com/acorn-io/baaah/pkg/lasso.(*Backend).Watch.func1({0x140003c0150?, 0x62bc2e2f?}, {0x101e9fe88?, 0x140006ce000?})
        /Users/anhtran/go/pkg/mod/github.com/acorn-io/[email protected]/pkg/lasso/backend.go:64 +0x70
github.com/rancher/lasso/pkg/controller.SharedControllerHandlerFunc.OnChange(0x14000073438?, {0x140003c0150?, 0x140000733a8?}, {0x101e9fe88?, 0x140006ce000?})
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/sharedcontroller.go:29 +0x44
github.com/rancher/lasso/pkg/controller.(*SharedHandler).OnChange(0x14000888730, {0x140003c0150, 0x22}, {0x101e9fe88, 0x140006ce000})
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/sharedhandler.go:75 +0x240
github.com/rancher/lasso/pkg/controller.(*controller).syncHandler(0x1400043a000, {0x140003c0150, 0x22})
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/controller.go:233 +0xa0
github.com/rancher/lasso/pkg/controller.(*controller).processSingleItem(0x1400043a000, {0x101cc3ec0?, 0x140003ae120?})
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/controller.go:214 +0xbc
github.com/rancher/lasso/pkg/controller.(*controller).processNextWorkItem(0x1400043a000)
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/controller.go:191 +0x48
github.com/rancher/lasso/pkg/controller.(*controller).runWorker(0x14000073688?)
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/controller.go:180 +0x2c
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x36b8?)
        /Users/anhtran/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x44
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x6?, {0x101e9bc98, 0x1400071b1a0}, 0x1, 0x140004ea120)
        /Users/anhtran/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x94
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x140008b6300?, 0x3b9aca00, 0x0, 0x90?, 0x0?)
        /Users/anhtran/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x84
k8s.io/apimachinery/pkg/util/wait.Until(0x7?, 0x14000073770?, 0x101e96b78?)
        /Users/anhtran/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x2c
created by github.com/rancher/lasso/pkg/controller.(*controller).run
        /Users/anhtran/go/pkg/mod/github.com/rancher/[email protected]/pkg/controller/controller.go:148 +0x2b8
panic: reflect: call of reflect.Value.Interface on zero Value [recovered]
        panic: reflect: call of reflect.Value.Interface on zero Value

Children of objects handled in Response are pruned, although they aren't touched by the Router

Context

I am using baaah to manage Tekton TaskRun objects.

When baaah creates those TaskRuns, they are automatically labeled with apply.acorn.io/hash, as expected.

Problem

Tekton propagates labels from the parent TaskRun to its children Pods, but baaah uses the aforementioned label to determine whether it should prune objects, and ends up pruning Pods which it doesn't manage.

Demonstration

Here is a debug session as an illustration, where the code snippets correspond to the breakpoints I set:

baaah/pkg/apply/desiredset_process.go

Lines 184 to 187 in 5d519d2

    
           existing, err := a.list(gvk, set, objs) 
        
           if err != nil { 
        
           	return fmt.Errorf("failed to list %s for %s: %w", gvk, debugID, err) 
        
           }

(dlv) print set
k8s.io/apimachinery/pkg/labels.Selector(k8s.io/apimachinery/pkg/labels.internalSelector) [
        {
                key: "apply.acorn.io/hash",
                operator: "=",
                strValues: []string len: 1, cap: 1, [
                        "576cd23303466727e7fab759412925c6f2bf8934",
                ],},
]

(dlv) print existing
map[github.com/acorn-io/baaah/pkg/apply/objectset.ObjectKey]sigs.k8s.io/controller-runtime/pkg/client.Object [
        {Name: "pod-managed-by-tekton", Namespace: "default"}: *k8s.io/api/core/v1.Pod {
                TypeMeta: (*"k8s.io/apimachinery/pkg/apis/meta/v1.TypeMeta")(0xc000a3a000),
                ObjectMeta: (*"k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta")(0xc000a3a020),
                Spec: (*"k8s.io/api/core/v1.PodSpec")(0xc000a3a108),
                Status: (*"k8s.io/api/core/v1.PodStatus")(0xc000a3a328),},
        {Name: "pod-managed-by-baaah", Namespace: "default"}: *k8s.io/api/core/v1.Pod {
                TypeMeta: (*"k8s.io/apimachinery/pkg/apis/meta/v1.TypeMeta")(0xc000a3a428),
                ObjectMeta: (*"k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta")(0xc000a3a448),
                Spec: (*"k8s.io/api/core/v1.PodSpec")(0xc000a3a530),
                Status: (*"k8s.io/api/core/v1.PodStatus")(0xc000a3a750),},
]

baaah/pkg/apply/desiredset_process.go

Lines 189 to 193 in 5d519d2

    
           var toReplace []objectset.ObjectKey 
        
           toCreate, toDelete, toUpdate := compareSets(existing, objs) 
        
           // check for resources in the objectset but under a different version of the same group/kind 
        
           toDelete = a.filterCrossVersion(allObjs, gvk, toDelete)

(dlv) print toDelete
[]github.com/acorn-io/baaah/pkg/apply/objectset.ObjectKey len: 1, cap: 1, [
        {
                Name: "pod-managed-by-tekton",
                Namespace: "default",},
]

As seen above, baaah matches on the Pod managed by Tekton, and ends up pruning that Pod, despite the Router's client never ever interacting with that particular Pod.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

gomod

go.mod

go 1.18

github.com/google/uuid v1.1.2

github.com/moby/locker v1.0.1

github.com/rancher/lasso v0.0.0-20220412224715-5f3517291ad4@5f3517291ad4

github.com/rancher/lasso/controller-runtime v0.0.0-20220412224715-5f3517291ad4@5f3517291ad4

github.com/sirupsen/logrus v1.8.1

github.com/stretchr/testify v1.7.0

golang.org/x/exp v0.0.0-20220428152302-39d4317da171@39d4317da171

k8s.io/api v0.23.6

k8s.io/apimachinery v0.23.6

k8s.io/client-go v0.23.6

sigs.k8s.io/controller-runtime v0.11.2

sigs.k8s.io/controller-tools v0.8.0

sigs.k8s.io/yaml v1.3.0

Check this box to trigger a request for Renovate to run again on this repository

Router client doesnt work for non-namespaced resources

In here:

baaah/pkg/router/client.go

Lines 126 to 141 in 91ce7be

    
           func (a *reader) List(ctx context.Context, list kclient.ObjectList, opts ...kclient.ListOption) error { 
        
           	listOpt := &kclient.ListOptions{} 
        
           	for _, opt := range opts { 
        
           		opt.ApplyToList(listOpt) 
        
           	} 
        
           	if listOpt.Namespace == "" { 
        
           		listOpt.Namespace = a.defaultNamespace 
        
           	} 
        
           	if err := a.registry.Watch(list, listOpt.Namespace, "", listOpt.LabelSelector); err != nil { 
        
           		return err 
        
           	} 
        
           	return a.client.List(ctx, list, listOpt) 
        
           }

the namespace is set to defaultNamespace if no namespace has been provided.

I was trying to use this to list Nodes, which are not namespaced. This causes the call to break and is inconsistent with how other implementations of the Reader interface behave.

tester package retrieves no objects for List when not providing namespace

func (c *Client) List(ctx context.Context, objList kclient.ObjectList, opts ...kclient.ListOption) error {

...

var ns string
	if listOpts.Namespace != "" {
		ns = listOpts.Namespace
	}

	// put objects into a map because c.objects() returns both created and updated objects, with updates coming after
	// created. this will ensure the last object in is what is returned
	resultObjs := make(map[string]runtime.Object)
	for _, testObj := range c.objects() {
		if testObj.GetNamespace() != ns {
			continue
		}

if a namespace is not provided in ListOptions, it is set to an empty string, which will not match testObj.GetNamespace(), resulting in no items fetched.

	existing, err := a.list(gvk, set, objs)
	if err != nil {
	return fmt.Errorf("failed to list %s for %s: %w", gvk, debugID, err)
	}

	var toReplace []objectset.ObjectKey
	toCreate, toDelete, toUpdate := compareSets(existing, objs)

	// check for resources in the objectset but under a different version of the same group/kind
	toDelete = a.filterCrossVersion(allObjs, gvk, toDelete)

	func (a *reader) List(ctx context.Context, list kclient.ObjectList, opts ...kclient.ListOption) error {
	listOpt := &kclient.ListOptions{}
	for _, opt := range opts {
	opt.ApplyToList(listOpt)
	}

	if listOpt.Namespace == "" {
	listOpt.Namespace = a.defaultNamespace
	}

	if err := a.registry.Watch(list, listOpt.Namespace, "", listOpt.LabelSelector); err != nil {
	return err
	}

	return a.client.List(ctx, list, listOpt)
	}

acorn-io / baaah Goto Github PK

baaah's People

Contributors

Stargazers

Watchers

Forkers

baaah's Issues

Context

Problem

Demonstration

Open

Detected dependencies

Recommend Projects

Recommend Topics

Recommend Org