Comments (6)
Can you capture the logs shown by the kubelet on the master node? Also, if you could get the node object's status? Using kubectl get node ${master-node-name} -o yaml and then paste the Status field.
from ovn-kubernetes.
In master node, we don't add CNI plugin. We assumed that you don't run kubelet on master. It looks like that may be wrong.
from ovn-kubernetes.
I'm prototyping a cluster based on kubeadm
and the master(s) run kubelet.
from ovn-kubernetes.
I was able to get my kubeadm
generated master deployed by ensuring the CNI config is present:
sudo mkdir -p /etc/cni/net.d
echo '{"name":"ovn-kubernetes", "type":"ovn-k8s-cni-overlay"}' | sudo tee /etc/cni/net.d/10-ovn-kubernetes.conf
sudo systemctl restart kubelet
I'm not able to get pods to deploy though:
Warning FailedCreatePodSandBox 40s (x52 over 20m) kubelet, worker-01 Failed create pod sandbox: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "kube-dns-86f4d74b45-77ktd_kube-system" network: failed to get pod annotation - timed out waiting for the condition
It looks like something is ignoring the kubeconfig and trying to hit the k8s api on localhost:8080
:
# kubelet logs
May 04 15:23:06 worker-01 kubelet[23894]: time="2018-05-04T15:23:06-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:07 worker-01 kubelet[23894]: time="2018-05-04T15:23:07-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:09 worker-01 kubelet[23894]: time="2018-05-04T15:23:09-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:11 worker-01 kubelet[23894]: time="2018-05-04T15:23:11-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:14 worker-01 kubelet[23894]: time="2018-05-04T15:23:14-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:20 worker-01 kubelet[23894]: time="2018-05-04T15:23:20-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:28 worker-01 kubelet[23894]: time="2018-05-04T15:23:28-04:00" level=warning msg="Error while obtaining pod annotations - Get http://localhost:8080/api/v1/namespaces/kube-system/pods/kube-dns-86f4d74b45-4d7mk: dial tcp [::1]:8080: connect: connection refused"
May 04 15:23:28 worker-01 kubelet[23894]: E0504 15:23:28.372042 23894 cni.go:259] Error adding network: failed to get pod annotation - timed out waiting for the condition
May 04 15:23:28 worker-01 kubelet[23894]: E0504 15:23:28.372097 23894 cni.go:227] Error while adding to cni network: failed to get pod annotation - timed out waiting for the condition
May 04 15:23:28 worker-01 kubelet[23894]: E0504 15:23:28.597701 23894 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "kube-dns-86f4d74b45-4d7mk_kube-system" network: failed to get pod annotation - timed out waiting for the condition
May 04 15:23:28 worker-01 kubelet[23894]: E0504 15:23:28.597765 23894 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "kube-dns-86f4d74b45-4d7mk_kube-system(a943c220-4fce-11e8-b283-0050569015cc)" failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "kube-dns-86f4d74b45-4d7mk_kube-system" network: failed to get pod annotation - timed out waiting for the condition
May 04 15:23:28 worker-01 kubelet[23894]: E0504 15:23:28.597787 23894 kuberuntime_manager.go:646] createPodSandbox for pod "kube-dns-86f4d74b45-4d7mk_kube-system(a943c220-4fce-11e8-b283-0050569015cc)" failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "kube-dns-86f4d74b45-4d7mk_kube-system" network: failed to get pod annotation - timed out waiting for the condition
May 04 15:23:28 worker-01 kubelet[23894]: E0504 15:23:28.597853 23894 pod_workers.go:186] Error syncing pod a943c220-4fce-11e8-b283-0050569015cc ("kube-dns-86f4d74b45-4d7mk_kube-system(a943c220-4fce-11e8-b283-0050569015cc)"), skipping: failed to "CreatePodSandbox" for "kube-dns-86f4d74b45-4d7mk_kube-system(a943c220-4fce-11e8-b283-0050569015cc)" with CreatePodSandboxError: "CreatePodSandbox for pod \"kube-dns-86f4d74b45-4d7mk_kube-system(a943c220-4fce-11e8-b283-0050569015cc)\" failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod \"kube-dns-86f4d74b45-4d7mk_kube-system\" network: failed to get pod annotation - timed out waiting for the condition"
May 04 15:23:28 worker-01 kubelet[23894]: W0504 15:23:28.679914 23894 pod_container_deletor.go:77] Container "50e0e24bc592c796b5d0126e1d9f607ab712b21a0b1b7d483539327488c6a3ec" not found in pod's containers
from ovn-kubernetes.
That is a good trick. Thanks @nlowe.
What I try to do is run ovnkube --init-node on the master node also. So that node also gets treated as a regular node.
from ovn-kubernetes.
This is fixed in master node. One needs to pass "init-node" as well as "init-master" to ovnkube as described in the README file.
from ovn-kubernetes.
Related Issues (20)
- Fix warnings and errors in go report card for ovn kubernetes
- Adding our project to artifact hub so that helm install is listed
- Bump CNI version from 0.4.0 to 1.0
- Make EIP node update event handling better
- Libovsdb docs; client and server side indexing
- Document all annotations, labels used by ovn kubernetes across all features
- Add templates for writing OVNKubernetesEnhancementProposal (OKEP) upstream
- e2e tests: enable kind-helm lane for ovn-ic
- Enhance EgressQoS CR to leverage entire OVN's QoS feature HOT 1
- unit tests Data race with level-driven informers not waiting for shut down HOT 1
- node deletion results stale lsps and IP leaking on layer2/localnet networks HOT 1
- UT Flake: `handles a HO node is switched to a OVN node` is flaking HOT 3
- Flake e2e: ACL Logging for NetworkPolicy when the namespace's ACL logging annotation is updated
- Load Balancer Service Tests with MetalLB [It] Should ensure load balancer service works with 0 node ports when ETP=local
- Cleanup Hardware Offload docs
- Cleanup DPU Support/Acceleration docs
- Cleanup Kubevirt Live Migration docs HOT 2
- Cleanup MultiNetworking Docs HOT 1
- Cleanup DNS name resolver docs HOT 1
- Add proper docs for observability, grafana dashboards, metrics
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ovn-kubernetes.