bowei / k8s-custom-iptables Goto Github PK

In this example are we making routing rule changes to the underlying k8s nodes?

I am trying to get custom ip tables to work for Google Memorystore from my non IP-Aliased k8s cluster.

Do I put TARGETS="{cluster CIDR} {MemoryStore CIDR}"

eg

TARGETS="10.0.0.0/29 10.78.0.0/14" 

or just

TARGETS="10.0.0.0/29" 

where 10.78.0.0/14 is my cluster CIDR and 10.0.0.0/29 is my memorystore CIDR?

I think im misunderstanding how it works

When using this script to create nat rules in a kubernetes cluster. I'm noticing that each node is receiving many of the same rules created by the script.

A node with 112 days uptime:

$ ~ # iptables -n -L -t nat | grep -ic custom
61

All 61 custom rules are the same - example:

...
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
...

Is this intended? The only thing that strikes me as an obvious problem is this:
https://github.com/bowei/k8s-custom-iptables/blob/master/run.sh#L37

I don't know if this was actually causing any problems but I noticed the logs for the k8s-custom-iptables pods had errors every time the configuration was being printed after I installed it with 2 CIDR ranges. I ran

TARGETS="10.0.0.0/29 10.0.0.8/29" ./install.sh

(These were the two ranges printed out by two Cloud Memorystore instances - I was following this documentation). In the logs I was getting:

This went away after I reconfigured to combine the blocks into 10.0.0.0/28

Sometimes the call to iptables fails with it printing that the resource was unavailable. In that case, the script should retry the call until it succeeds. Currently, it fails instead, which causes the pod to be restarted.

https://www.alpinelinux.org/posts/Docker-image-vulnerability-CVE-2019-5021.html

I can find this in an instace of the image I run in a cluster.

⇒  kubectl exec -it k8s-custom-iptables-xxx /bin/sh
/ # cat /etc/shadow | grep root
root:::0:::::

There are issues with daemon.yaml.

$ TARGETS="10.48.50.8/29" ./install.sh
configmap/k8s-custom-iptables unchanged
error: unable to recognize "daemon.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"

#12 -> fixes the Problem for me.