wearemolecule / route53-kubernetes Goto Github PK
View Code? Open in Web Editor NEW[DEPRECATED] Sync Kubernetes Services with AWS Route53
License: MIT License
[DEPRECATED] Sync Kubernetes Services with AWS Route53
License: MIT License
I created an IAM policy like so:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "route53:ListHostedZonesByName",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:DescribeLoadBalancers",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "route53:ChangeResourceRecordSets",
"Resource": "*"
}
]
}
and associated it with the same use that was used to create my kubernetes cluster.
However, when starting up the deployment of this application I get errors about IAM permissions:
service_listener.go:145] Couldn't get zone ID: Could not describe load balancer: AccessDenied: User: arn:aws:sts::201248577332:assumed-role/kubernetes-minion/i-704a92de is not authorized to perform: elasticloadbalancing:DescribeLoadBalancers
internal ELBs' DNS is prepended by internal-
, this line, that should retrieve the ELB name, doesn't work with that naming schema, because it returns internal
as the dns name
Hi in our organization we have setup different HostedZones for different nested levels of DNS. For example our HostedZones look like this:
HostedZone 1: example.com
HostedZone 2: stage.example.com
Unfortunately when I use route53 to create a DNS record for "kubernetes.stage.example.com" it wrongfully creates the record in the "example.com" HostedZone. As everything that ends with "stage.example.com" is handled by the "stage.example.com" HostedZone, this obviously doesn't work.
Would it be possible to make route53 choose the HostedZone that is most specific?
I suspect that my issue is also what is mentioned in this comment:
route53-kubernetes/service_listener.go
Line 161 in 5eb3615
https://github.com/wearemolecule/route53-kubernetes/blob/master/service_listener.go#L116
Currently the logic in the loop does not detect if the recordset for the ELB has already been created, causing the Go code to continuously make the ChangeResourceRecordSets
API call to AWS.
The script should detect if the correct recordset exists before making the call.
Would it be within scope of this project to support Ingress resources?
I'd love to use this project but in my setup I have a single LoadBalancer
service that needs multiple declared DNS entries pointing to it for use with Ingress -- each of these would be bound to an Ingress
resource. Ideally, I'd just annotate the Ingress
resource with a targeted LoadBalancer
service or something along those lines.
Hi! I can't seem to find this dependency in the k8s repo, and go get doesn't find it, so my compile fails. Not sure what I'm missing.
k8s.io/kubernetes/pkg/client/restclient
Any chance we could get documentation regarding how to run the route53-kubernetes rc/service? on kubernetes, perhaps a .yml file? thanks
I realize this would require starting to maintain state, which is a larger change.
It would be great to keep the metadata simple and not tied to specific cluster/DNS zone if we could provide only the hostname part of the FQDN, ie for FQDN myapp.k8s-1.5.example.com
the hostname part is myapp
.
Avoiding details specific to deployment are a good practice which motivated Persistent Volumes and Storage Classes that are considered infrastructure part in Kubernetes where the Persistent Volume Claims are closer to the application.
In a scenario where this would be useful for me is during blue/green upgrades of Kubernetes itself. Let's say if I have a cluster (created using kops) using domain k8s-1.5.example.com
and I want to move all my deployments and services over to new cluster under k8s-1.6.example.com
. With this feature I wouldn't have to update all service yaml definitions.
Hi,
Following the setup instructions i seem to be stuck with
2016-08-29T11:21:23.334391202Z I0829 11:21:23.334221 1 service_listener.go:28] Route53 Update Service
2016-08-29T11:21:23.334509248Z F0829 11:21:23.334311 1 service_listener.go:54] Couldn't set up tls transport: open /etc/kubernetes/ssl/ca.pem: no such file or directory
Where do i get the cert files from? What are they meant to be for? They don't exist on my master or minion servers?
I just found your project and was very excited to try it out, unfortunately it seems to always use just the domain as its base zone, instead of only using the hostname from the FQDN and using the rest as the zone.
So in my example i have traefik.k8s.int.example.com
, the hosted zone being k8s.int.example.com
, but it tries to use the zone example.com
.
I guess only using the hostname is a safe bet. Otherwise maybe get the hosted zones from Route53 and try to match the record to the zone that contains "the longest part" of the FQDN?
any ideas how i can install it to my existing kubernetes stack?
I would like to throw out the idea of making things work from a cluster that resides outside of AWS. This would be useful for organizations that have a multi-cloud setup where all DNS is in Route53.
The technical solution could involve having the AWS credentials as a secret. Load balancers in other services, like Azure, have a fixed IP address rather than a DNS name like AWS, so they would need the appropriate record type in Route53.
Thoughts ?
At my current job we setup route53 service names like this.
funservice.domainname.com -> (ALIAS/A) us-east-1-clustername-funservice.domainname.com
us-east-1-clustername-funservice.domainname.com -> (ALIAS/A)
So an service may have these names at a level up from each of the cluster specific ELB's.
funservice.domainname.com -> (ALIAS/A) us-east-1-a-funservice.domainname.com
funservice.domainname.com -> (ALIAS/A) us-east-1-b-funservice.domainname.com
funservice.domainname.com -> (ALIAS/A) us-west-2-a-funservice.domainname.com
I was thinking an optional annotation like 'domainNameParent' would allow us to have the cluster specific ELB get setup as well as the parent name that we expose to our users.
Having a single hardcoded hostname causes problems for us.
We want to be able to reuse a single service definition in different namespaces and clusters.
The kubernetes/kubernetes#21397 issue suggests a format for dynamically defining route53 hostnames using the format [service-name]-[namespace]-[hosted-zone].[zone]
. That would work for us but is very rigid.
If the domainName
annotation supported template fields then we'd be able to express our needs as something like: domainName: "{{.service}}-{{.namespace}}.mydomain.com"
.
(I think that would be simple to implement using text/template. I'd offer a PR but I've not done any Go programming yet.)
Hi,
If a service gets deleted in Kubernetes, then does the plugin automatically detect it and delete the corresponding Route 53 entry?
Thanks,
It would be useful to allow TLD assignment so a service can, for example, support github.com
and not just www.github.com
.
Support service account to talk to Kubernetes API.
Caveat: I'm new to golang so entirely possible I've making a noob mistake here but two hours into a small patch and I still can't seem to be able to get a working binary for this project. Any help/guidance would be appreciated!
Build steps (ubuntu 14.04):
glide install
make build
Runtime error:
./route53-kubernetes flag redefined: log_dir
panic: ./route53-kubernetes flag redefined: log_dir
goroutine 1 [running]:
panic(0x141dd00, 0xc82000b750)
/usr/lib/go/src/runtime/panic.go:464 +0x3e6
flag.(*FlagSet).Var(0xc8200160c0, 0x7f3538da56a0, 0xc82000b700, 0x19d0000, 0x7, 0x1b76fc0, 0x2f)
/usr/lib/go/src/flag/flag.go:776 +0x454
flag.(*FlagSet).StringVar(0xc8200160c0, 0xc82000b700, 0x19d0000, 0x7, 0x0, 0x0, 0x1b76fc0, 0x2f)
/usr/lib/go/src/flag/flag.go:679 +0xc7
flag.(*FlagSet).String(0xc8200160c0, 0x19d0000, 0x7, 0x0, 0x0, 0x1b76fc0, 0x2f, 0xc82000b6f0)
/usr/lib/go/src/flag/flag.go:692 +0x83
flag.String(0x19d0000, 0x7, 0x0, 0x0, 0x1b76fc0, 0x2f, 0x7f3538da5918)
/usr/lib/go/src/flag/flag.go:699 +0x5f
route53-kubernetes/vendor/k8s.io/kubernetes/vendor/github.com/golang/glog.init()
/home/ubuntu/go/src/route53-kubernetes/vendor/k8s.io/kubernetes/vendor/github.com/golang/glog/glog_file.go:41 +0x13e
route53-kubernetes/vendor/k8s.io/kubernetes/pkg/labels.init()
/home/ubuntu/go/src/route53-kubernetes/vendor/k8s.io/kubernetes/pkg/labels/selector.go:808 +0x5b
route53-kubernetes/vendor/k8s.io/kubernetes/pkg/api/unversioned.init()
/home/ubuntu/go/src/route53-kubernetes/vendor/k8s.io/kubernetes/pkg/api/unversioned/well_known_labels.go:30 +0x6f
route53-kubernetes/vendor/k8s.io/kubernetes/pkg/api.init()
/home/ubuntu/go/src/route53-kubernetes/vendor/k8s.io/kubernetes/pkg/api/types.go:2884 +0x64
main.init()
/home/ubuntu/go/src/route53-kubernetes/service_listener.go:252 +0x80
Seems like a conflict with vendored deps, so I tried the following:
glide install -v
make build
This time I get a build error:
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -o route53-kubernetes .
# route53-kubernetes/vendor/golang.org/x/oauth2/google
vendor/golang.org/x/oauth2/google/default.go:92: undefined: metadata.OnGCE
vendor/golang.org/x/oauth2/google/google.go:117: undefined: metadata.OnGCE
vendor/golang.org/x/oauth2/google/google.go:124: undefined: metadata.Get
make: *** [build] Error 2
Any ideas? (Working on a fix that would use the service account credential and eliminate the need to mount a volume/etc)
I've been trying to run the service on a kubernetes cluster (v1.2.4) but it seems to be failing:
Looking in the logs I get the following:
I0601 13:37:39.698519 1 service_listener.go:26] Route53 Update Service
I0601 13:37:39.699695 1 service_listener.go:64] Connected to kubernetes @ https://10.0.0.1:443
I0601 13:37:39.699822 1 service_listener.go:93] Starting Service Polling every 30s
F0601 13:37:39.721357 1 service_listener.go:97] Failed to list pods: the server has asked for the client to provide credentials (get services)
So it seems that the service is unable to contact the k8s API server.
Using the same credentials I am able to contact the API server from an external node.
I've been trying to run this as a Kubernetes service for a while now and it keeps failing with the following exception. This happens with both v1.1.0 and v1.1.1.
The node running the container has elasticloadbalancing:DescribeLoadBalancers
so I don't think it's permissions related. I was able to run that command manually on the same node with Python.
I0519 01:14:48.143918 1 service_listener.go:26] Route53 Update Service
I0519 01:14:48.145060 1 service_listener.go:64] Connected to kubernetes @ https://10.3.0.1:443
I0519 01:14:48.145168 1 service_listener.go:93] Starting Service Polling every 30s
I0519 01:14:48.243432 1 service_listener.go:100] Found 1 DNS services in all namespaces with selector "dns=route53"
I0519 01:14:48.243521 1 service_listener.go:115] Creating DNS for nginx service: XXXXX-XXXX.us-XXX.elb.amazonaws.com -> XXXX.YYYY.com
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x708590]
goroutine 1 [running]:
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/ec2metadata.(*EC2Metadata).GetMetadata(0x0, 0x14800c0, 0x19, 0x0, 0x0, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/ec2metadata/api.go:18 +0x1b0
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds.requestCredList(0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds/ec2_role_provider.go:133 +0x74
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds.(*EC2RoleProvider).Retrieve(0xc820212cc0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds/ec2_role_provider.go:89 +0x8b
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials.(*ChainProvider).Retrieve(0xc820212cf0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials/chain_provider.go:75 +0x122
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials.(*Credentials).Get(0xc82020ecc0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/credentials/credentials.go:185 +0x121
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/private/signer/v4.(*signer).sign(0xc820225118, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/private/signer/v4/v4.go:182 +0x3c5
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/private/signer/v4.Sign(0xc82000db80)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/private/signer/v4/v4.go:155 +0x1fa
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc82000dca8, 0xc82000db80)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/request/handlers.go:115 +0x9f
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/request.(*Request).Sign(0xc82000db80, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/request/request.go:212 +0xce
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc82000db80, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/aws/request/request.go:222 +0x54
github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/service/elb.(*ELB).DescribeLoadBalancers(0xc820024270, 0xc8202e9170, 0xc8202e5400, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/aws/aws-sdk-go/service/elb/api.go:608 +0x4f
main.hostedZoneId(0xc820024270, 0xc8202ddb80, 0x47, 0x0, 0x0, 0x0, 0x0)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/service_listener.go:178 +0x1ab
main.main()
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/service_listener.go:121 +0x1f68
goroutine 5 [chan receive]:
github.com/wearemolecule/route53-kubernetes/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x1b2f360)
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/golang/glog/glog.go:879 +0x67
created by github.com/wearemolecule/route53-kubernetes/vendor/github.com/golang/glog.init.1
/Users/iterion/Development/go/src/github.com/wearemolecule/route53-kubernetes/vendor/github.com/golang/glog/glog.go:410 +0x297
goroutine 17 [IO wait]:
net.runtime_pollWait(0x7f0c43192a28, 0x72, 0xc82000a130)
/usr/local/Cellar/go/1.5/libexec/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82020b870, 0x72, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82020b870, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82020b810, 0xc8201e4c00, 0x400, 0x400, 0x0, 0x7f0c43186050, 0xc82000a130)
/usr/local/Cellar/go/1.5/libexec/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8200242a0, 0xc8201e4c00, 0x400, 0x400, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/net/net.go:172 +0xe4
crypto/tls.(*block).readFromUntil(0xc820213650, 0x7f0c43192ae8, 0xc8200242a0, 0x5, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/crypto/tls/conn.go:455 +0xcc
crypto/tls.(*Conn).readRecord(0xc82000d8c0, 0x15fe417, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/crypto/tls/conn.go:540 +0x2d1
crypto/tls.(*Conn).Read(0xc82000d8c0, 0xc8201d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/crypto/tls/conn.go:901 +0x167
net/http.noteEOFReader.Read(0x7f0c43192fd8, 0xc82000d8c0, 0xc8201f1918, 0xc8201d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8202e6d20, 0xc8201d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
<autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202e2720)
/usr/local/Cellar/go/1.5/libexec/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202e2720, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.5/libexec/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201f18c0)
/usr/local/Cellar/go/1.5/libexec/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
/usr/local/Cellar/go/1.5/libexec/src/net/http/transport.go:685 +0xc78
goroutine 18 [select]:
net/http.(*persistConn).writeLoop(0xc8201f18c0)
/usr/local/Cellar/go/1.5/libexec/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
/usr/local/Cellar/go/1.5/libexec/src/net/http/transport.go:686 +0xc9d
As it seems to be related to the AWS SDK, I tried updating the dependencies and rebuilding, but that fails because of kubernetes/kubernetes#25823. Updating to latest Kubernetes errors when building on:
./service_listener.go:55: undefined: "route53-kubernetes/vendor/k8s.io/kubernetes/pkg/client/unversioned".Config
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.