bowei / k8s-custom-iptables Goto Github PK

How to add custom iptables rules to a Kubernetes cluster

License: Apache License 2.0

Makefile 14.14% Shell 75.81% Dockerfile 10.05%

k8s kubernetes nat iptables networking masquerade

k8s-custom-iptables's Introduction

k8s-custom-iptables

An example of how to add custom IP tables rules to a Kubernetes cluster. This collection of scripts creates a NAT (MASQ) rule for outbound traffic to a TARGETS CIDR range(s) given to the script.

Installing rules into the cluster.

Install the daemonset that configures the cluster to NAT an IP range.

TARGETS="1.2.3.4/24 4.5.6.7/16" ./install.sh

Uninstall rules from the cluster.

Uninstall the IP tables rules from the cluster.

./uninstall.sh

Configuring

The configuration for which ranges are NAT'd are in the k8s-custom-iptables ConfigMap. Values can be changed via kubectl edit cm/k8s-custom-iptables:

apiVersion: v1
kind: ConfigMap
metadata:
  name: k8s-custom-iptables
data:
  nat.rules: "10.0.0.0/24 192.168.0.0/16"

Creating and pushing the image.

REGISTRY=gcr.io/my-registry make

k8s-custom-iptables's People

Contributors

Stargazers

Watchers

k8s-custom-iptables's Issues

error: unable to recognize "daemon.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"

There are issues with daemon.yaml.

$ TARGETS="10.48.50.8/29" ./install.sh
configmap/k8s-custom-iptables unchanged
error: unable to recognize "daemon.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"

#12 -> fixes the Problem for me.

duplicate iptables nat rules

When using this script to create nat rules in a kubernetes cluster. I'm noticing that each node is receiving many of the same rules created by the script.

A node with 112 days uptime:

$ ~ # iptables -n -L -t nat | grep -ic custom
61

All 61 custom rules are the same - example:

...
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
MASQUERADE  all  --  0.0.0.0/0            10.0.x.x/29          /* custom-iptables-1534894989: 10.0.x.x/29 */
...

Is this intended? The only thing that strikes me as an obvious problem is this:
https://github.com/bowei/k8s-custom-iptables/blob/master/run.sh#L37

what does the TARGETS env var do?

I am trying to get custom ip tables to work for Google Memorystore from my non IP-Aliased k8s cluster.

Do I put TARGETS="{cluster CIDR} {MemoryStore CIDR}"

TARGETS="10.0.0.0/29 10.78.0.0/14"

or just

TARGETS="10.0.0.0/29"

where 10.78.0.0/14 is my cluster CIDR and 10.0.0.0/29 is my memorystore CIDR?

I think im misunderstanding how it works

iptables fails with resource unavailable

Sometimes the call to iptables fails with it printing that the resource was unavailable. In that case, the script should retry the call until it succeeds. Currently, it fails instead, which causes the pod to be restarted.

Update docker image in gcr repo to mitigate CVE-2019-5021

https://www.alpinelinux.org/posts/Docker-image-vulnerability-CVE-2019-5021.html

I can find this in an instace of the image I run in a cluster.

⇒  kubectl exec -it k8s-custom-iptables-xxx /bin/sh
/ # cat /etc/shadow | grep root
root:::0:::::

errors in the logs after installing 2 contiguous cidr ranges

I don't know if this was actually causing any problems but I noticed the logs for the k8s-custom-iptables pods had errors every time the configuration was being printed after I installed it with 2 CIDR ranges. I ran

TARGETS="10.0.0.0/29 10.0.0.8/29" ./install.sh

(These were the two ranges printed out by two Cloud Memorystore instances - I was following this documentation). In the logs I was getting:

This went away after I reconfigured to combine the blocks into 10.0.0.0/28

How does this work?

In this example are we making routing rule changes to the underlying k8s nodes?