avinetworks / avi-helm-charts Goto Github PK

hello,experts. follow rancher guide I use helm deployed rancher at TKGs K8s Cluster, pls refer to command line.
helm install rancher rancher-latest/rancher --version 2.5.9 --namespace cattle-system --set hostname={FQDN Name} --set replicas=1 ingress.tls.source=secret --set privateCA=true --set bootstrapPassword={Password}
I find the pod/Service/Ingress is running , pls refer to the pic

but I couldn't asscess the URL, I check the ingress and got the error message. pls help me to fixed the issue. thanks a lot

In statefulset.yaml line 19~21, imagePullSecrets's value is wrong, it should be - name: {{ toYaml . }}

    {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}

Fix:

    {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        - name: {{  toYaml . }}
    {{- end }}

In the AKO chart the image tag is hard coded:

image: "{{ .Values.image.repository }}:{{ .Chart.AppVersion }}"

We rebuild all of our open source images in our own repository and apply a build date suffix. We can't use the chart as it is without modifying it.

Standard practice is to allow the image tag to be overridden and have the image tag supplied in the default values.yaml file. Like this:

image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"

Is this still being published? You have a change log with 1.4 but I don't see those artifacts and docker only shows 1.3.1:

https://hub.docker.com/r/avinetworks/ako/tags?page=1&ordering=last_updated

I deployed this helm chart and it creates a service account, which doesn't have permissions to do stuff

Failed to list *v1.ConfigMap: Unauthorized

I created a role binding and just bound it to cluster admin, but thats probably overkill:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ak-sabinding
  namespace: avi-system
subjects:
  - kind: ServiceAccount
    name: ako-sa
    namespace: avi-system
roleRef:
  kind: Role
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

Whats your recommendation on role permissions?

Hello all.

we are having a strange issue happening in our lab and are looking for some help trying to pin it down.

AKO Version - 1.7.2
AVI Controller Version = 21.1.4-9210

Issue Description - when doing CI/CD deployments of our clusters using AKO/AVI, we see that only 1 out of every 10 deployments or so "succeeds" in bringing up the VIP - meaning that its reachable beyond the cluster nodes themselves. When it "works" we see that the VSVIP, Virtual Service, Pool and SE-GROUP Engine are brought up and the VIP is routables without issue.

When it doesnt work - we see that that the VSVIP is and Virtual Service are created however, we see a couple different errors in the AVI controller:

• Pools dont exist
• Unable/waiting for SE Creation (and then failing on the POOLs)

Conditions - between each CI/CD run, we remove all traces of the VSVIP, Service Engine (not the group, but the SE that was broughtup), and traces of the Static Routes that were used from the Worker nodes internal network to the IPs of ETH0 of those nodes (for routing in our AVI).

• if we wait a "while" a day or so - we can see that eventually this "works again"

• We observe that our AKO CNI is applied and the AKO-0 POD comes up and is able to reach the AVI Controller

• We observe that the VSVIP and Virtual Service is created as the LB are created in line.

Errors from AKO Logs
When this issue occurs, we see Errors in the AKO PODs complaining that the Service Already exists - when as outlined, all traces (that are available to us) have been removed. We have look and implemented some code check using the go AVI SDK (to list and remove VSVIP, Virtual Service, etc) - however there is no API that lets you remove the Static Routes.

We see that AKO doesnt throw errors, but AVI complains about POOLS being down and such.

Our Thoughts
it would seem that AVI Controller is 'cacheing or holding" information about previous setups somewhere that we cannot clean out, this makes doing automated end to end testing very problematic, since we never "know" when the AVI /AKO combination will work or not.

Our setup/configuration is static - doesnt change every deployment and yet, as observed, 1/10 times we deploy the path "opens correctly' - other times we have to bounce AKO pods, remove and read things in AVI controller side and its really "hard" to pin down what is the cause of this.

Our need is that we can deploy our clusters (end to end including the LB setusp for our vips) are part of our e2e testing cycle, however this instability is creating some havoc as you can imagine.

Any thoughts on where this might be "hidden" so we can remove the memory of a previous deploy and our LBs come up without issue?

NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                                      AGE
istio-ingressgateway   LoadBalancer   10.43.108.67   <pending>     15021:30430/TCP,80:31812/TCP,443:31015/TCP   19m

My ip is stuck pending despite what looks to be fully functional vips in the AVI ui

Now we did recently change environments where in my POC I had admin, and in this env I do not. The only thing that's changed is tenants per cluster.

I think it's related to this error from the controller.

2021-06-21T20:36:12.597Z ERROR status/svc_status.go:37 Service hostname not found for service %v status update[istio-system/istio-ingressgateway]

However, notice %v if I recall, %v is object reflection. It just says take what ever this is and print it. So it's an array, string, int etc what ever print it as it's ASCII representaion. So something isn't right here. I can't actually tell what this wants. Idk what it means by service hostname

According to hostrule.md, enableVirtualhost: true or enableVirtualHost: true is configured, and I have received an API error and cannot recognize this field.

Specifically, I'm having difficulty understanding how to create an avi ingress object that passes back to a service type of node port.

Your read me: https://github.com/avinetworks/avi-helm-charts/blob/master/docs/AKO/README.md

has Using NodePort mode

But your values file does not reflect this. Additionally you have spare bits floating around like:

service:
  type: ClusterIp
  port: 80

But I don't think this is used
https://github.com/avinetworks/avi-helm-charts/blob/master/charts/stable/ako/values.yaml
Can you provide an example values file that will pass back to a service type of node port?

I tried to install and configure AKO and AKO Operator in K8S Cluster on Oralce Cloud Infrastructure.
I can't match IPAM Profile for select OCI Network dans network IP subnet. Did the AKO is compatible to Public Cloud IPAM Solution ?

To reproduce:

Install 1.3.1 via helm. Then remove it via helm uninstall. There will be varying dangling assets.

From here, try to delete the name space of avi-system In particular it seems that ako-config hard hangs on removal

ku get akoconfig.ako.vmware.com
NAME         AGE
ako-config   14h

This has been hung for 14 hours now

For hostnames, the standard limit for length is 253 octets and 63 chars per dns label.

Avi supports 63 chars per label and 253 octets max for the entire hostname(including dots). And throws the following error when the hostname exceeds 253 chars (254 in this case).

The Ingress "avi-ingress-long-hostname" is invalid: spec.rules[0].host: Invalid value: "abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz1.abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz1.abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz1.abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklm.mydomain.com": must be no more than 253 characters

But looks like if the if the DNS label exceeds 63 chars, no relevant error is shown, the VS is never created. There are some errors in AKO pod, but none of them indicate the 63 chars limit. The Ingress simply doesn't get any IP address and no events are generated on k8s side. AKO should bubble-up this error for end-users.

Let me know if you need more details. Thanks!

Version 1.3.1 of the chart

Kube versions:

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.6", GitCommit:"fbf646b339dc52336b55d8ec85c181981b86331a", GitTreeState:"clean", BuildDate:"2020-12-18T12:01:36Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

This could be related to my other bug. According to these docs:
https://github.com/avinetworks/avi-helm-charts/blob/master/docs/AKO/objects.md
See: NodePort Mode

I have a service:

apiVersion: v1
kind: Service
metadata:
  annotations:
    field.cattle.io/publicEndpoints: '[{"addresses":["10.2.215.244"],"port":31370,"protocol":"TCP","serviceName":"istio-system:istio-ingressgateway","allNodes":true},{"addresses":["10.2.215.244"],"port":31380,"protocol":"TCP","serviceName":"istio-system:istio-ingressgateway","allNodes":true},{"addresses":["10.2.215.244"],"port":30238,"protocol":"TCP","serviceName":"istio-system:istio-ingressgateway","allNodes":true}]'
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"istio-ingressgateway","install.operator.istio.io/owning-resource":"istio-controlplane","install.operator.istio.io/owning-resource-namespace":"istio-system","istio":"ingressgateway","istio.io/rev":"default","operator.istio.io/component":"IngressGateways","operator.istio.io/managed":"Reconcile","operator.istio.io/version":"1.7.6","release":"istio"},"name":"istio-ingressgateway","namespace":"istio-system"},"spec":{"ports":[{"name":"status-port","nodePort":31370,"port":15021,"targetPort":15021},{"name":"http2","nodePort":31380,"port":80,"targetPort":8080},{"name":"https","port":443,"targetPort":8443}],"selector":{"app":"istio-ingressgateway","istio":"ingressgateway"},"type":"NodePort"}}
  creationTimestamp: "2021-01-21T17:11:45Z"
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: istio-controlplane
    install.operator.istio.io/owning-resource-namespace: istio-system
    istio: ingressgateway
    istio.io/rev: default
    operator.istio.io/component: IngressGateways
    operator.istio.io/managed: Reconcile
    operator.istio.io/version: 1.7.6
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
  resourceVersion: "250491"
  selfLink: /api/v1/namespaces/istio-system/services/istio-ingressgateway
  uid: 7c075802-1aaa-40ce-bb30-5dd584eac889
spec:
  clusterIP: 10.43.253.225
  externalTrafficPolicy: Cluster
  ports:
  - name: status-port
    nodePort: 31370
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: http2
    nodePort: 31380
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    nodePort: 30238
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

It creates the pools:

But no servers in those pools. Type of load balancer works fine.

closed

All of this stuff feels super esoteric, then just the general layering of corporate access from our side makes things difficult as well. Some one handed me an avi, node, controller, whatever you want to call it. I have an avi portal I can log into.

I've followed the basic setup instructions, my service engine group is 20.1.2-9171:

I've deployed version 1.2.3, showing compatibility with 20.1.1 as a minimum requirement (but I might be looking at the wrong thing, everything is an acronym and I'm not even sure most of them are explained, I have large gaps in my understanding)

The error I get:

2020-12-16T19:46:47.460Z	WARN	cache/controller_obj_cache.go:2512	Setting labels on Service Engine Group :Default-Group failed with error :Encountered an error on PUT request to URL https://[REDACT]//api/serviceenginegroup/serviceenginegroup-ac49cc90-a64f-4811-957f-ad589afa735e: HTTP code: 400; error from Avi: map[error:Incoming request (v18_2_10) has field (labels) introduced in later versions(v20_1_1).]. Expected Labels: [{"key":"clustername","value":"single-node"}]