Code Monkey home page Code Monkey logo

hadoop's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

hadoop's Issues

with init script, pod can not connect the service.

Env:Vagrant + coreos * 2
K8s version: 1.2.0
CoreOS version: Linux version 4.6.0-coreos ([email protected]) (gcc version 4.9.3 (Gentoo Hardened 4.9.3 p1.3, pie-0.6.3) ) #2 SMP Fri Jun 3 23:44:22 UTC 2016

  • svc info
yancey@ yancey-macbook k8s$kubectl get svc
NAME               CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
hadoop-zookeeper   10.0.0.120   <none>        2181/TCP   1h
kubernetes         10.0.0.1     <none>        443/TCP    1h
  • my pod can not telnet to the kubernets service
yancey@ yancey-macbook k8s$kubectl exec  hadoop-zookeeper-x96rj telnet 10.0.0.1 443
telnet: can't connect to remote host (10.0.0.1): Connection refused
error: error executing remote command: Error executing command in container: Error executing in Docker Container: 1
  • ifconfig && iptables info
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.1.4.1  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::42:bcff:fef3:35ab  prefixlen 64  scopeid 0x20<link>
        ether 02:42:bc:f3:35:ab  txqueuelen 0  (Ethernet)
        RX packets 159  bytes 12380 (12.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 21  bytes 1242 (1.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.2.15  netmask 255.255.255.0  broadcast 10.0.2.255
        inet6 fe80::a00:27ff:fea7:5e9e  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:a7:5e:9e  txqueuelen 1000  (Ethernet)
        RX packets 97026  bytes 76268827 (72.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 45284  bytes 3009253 (2.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.8.102  netmask 255.255.255.0  broadcast 172.17.8.255
        inet6 fe80::a00:27ff:feda:478f  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:da:47:8f  txqueuelen 1000  (Ethernet)
        RX packets 39909  bytes 4961258 (4.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 40458  bytes 4070934 (3.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.1.4.0  netmask 255.255.0.0  broadcast 0.0.0.0
        inet6 fe80::b3:c3ff:fe13:4d94  prefixlen 64  scopeid 0x20<link>
        ether 02:b3:c3:13:4d:94  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 8 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 2360  bytes 122960 (120.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2360  bytes 122960 (120.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth054ea6f: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet6 fe80::60c5:d4ff:fe2e:b234  prefixlen 64  scopeid 0x20<link>
        ether 62:c5:d4:2e:b2:34  txqueuelen 0  (Ethernet)
        RX packets 159  bytes 14606 (14.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 29  bytes 1890 (1.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

core-v2-02 init # iptables-save -t nat
...
-A KUBE-SEP-OGNOLD2JUSLFPOMZ -s 10.0.2.15/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-OGNOLD2JUSLFPOMZ -p tcp -m comment --comment "default/kubernetes:https" -m tcp -j DNAT --to-destination 10.0.2.15:6443
-A KUBE-SERVICES -d 10.0.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
...
  • I think the address in iptables 10.0.2.15 should be insted of eth1(172.17.8.102), but I can not find the solution.

RM can not find the host of hadoop-slave-xxx

I'm tring to run the example WordCount, but the app state alwasy ACCEPTED.
In RM log:

16/05/08 15:56:36 ERROR scheduler.SchedulerApplicationAttempt: Error trying to assign container token and NM token to an allocated container container_1462716844418_0001_01_000001
java.lang.IllegalArgumentException: java.net.UnknownHostException: hadoop-slave-qjzbu
    at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374)
    at org.apache.hadoop.yarn.server.utils.BuilderUtils.newContainerToken(BuilderUtils.java:256)
    at org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTokenSecretManager.createContainerToken(RMContainerTokenSecretManager.java:220)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.pullNewlyAllocatedContainersAndNMTokens(SchedulerApplicationAttempt.java:448)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.allocate(FairScheduler.java:921)

It looks like RM node can not connect the pod of NM node,such as hadoop-slave-xxxx.

So my question is :

I can use dns service with the component name such as hadoop-master, but when the hadoop-master want to connect the specific pod, such as hadoop-slave-xxx, it can not find the ip.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.