部署MVP遇到的问题 about kuscia HOT 13 CLOSED

安转部署包使用的是什么操作系统？

from kuscia.

安转部署包使用的是什么操作系统？

我在centos7的linux环境部署

from kuscia.

安转部署包使用的是什么操作系统？

我在centos7的linux环境部署

感谢反馈，可以按照如下方法提供下日志：
查看Master容器日志：
1 通过 docker ps 命令，查看kuscia-master 容器名称
docker ps

容器名称一般为 {user}-kuscia-lite-alice ，类似 tom-kuscia-master
2 执行如下命令进入到docker容器中：
docker exec -it tom-kuscia-master bash
3 查看 kuscia日志
cd /home/kuscia/var/logs

查看 k3s.log 、 kuscia.log

from kuscia.

安转部包使用的是什么操作系统？

我在centos7的linux环境部署

感谢反馈，可以按照如下方法提供下日志： 查看Master容器日志： 1 通过 docker ps 命令，查看kuscia-master 容器名称 docker ps 容器名称一般为 {user}-kuscia-lite-alice ，类似 tom-kuscia-master 2 执行如下命令进入到docker容器中： docker exec -it tom-kuscia-master bash 3 查看 kuscia日志 cd /home/kuscia/var/logs 查看 k3s.log 、 kuscia.log

安转部署包使用的是什么操作系统？

我在centos7的linux环境部署

感谢反馈，可以按照如下方法提供下日志： 查看Master容器日志： 1 通过 docker ps 命令，查看kuscia-master 容器名称 docker ps 容器名称一般为 {user}-kuscia-lite-alice ，类似 tom-kuscia-master 2 执行如下命令进入到docker容器中： docker exec -it tom-kuscia-master bash 3 查看 kuscia日志 cd /home/kuscia/var/logs 查看 k3s.log 、 kuscia.log

这是我运行的指令 grep "ERROR" kuscia.log
可以看到这些内容2023-07-26 10:29:28.093 ERROR modules/k3s.go:56 open /home/kuscia/var/k3s/server/tls/server-ca.crt: no such file or directory
2023-07-26 10:29:29.157 ERROR modules/k3s.go:97 Get ready err:Get "https://127.0.0.1:6443/readyz": dial tcp 127.0.0.1:6443: connect: connection refused
2023-07-26 10:30:11.579 ERROR modules/k3s.go:308 apply /home/kuscia/crds/v1alpha1/kuscia.secretflow_appimages.yaml err:exit status 1
2023-07-26 10:30:11.580 ERROR modules/k3s.go:205 exit status 1
2023-07-26 10:30:11.597 ERROR modules/k3s.go:196 context had done, no need to wait to restart
2023-07-26 10:30:12.796 ERROR modules/k3s.go:308 apply /home/kuscia/etc/conf/kuscia-clusterrole.yaml err:exit status 1
2023-07-26 10:30:12.796 ERROR modules/k3s.go:209 exit status 1
2023-07-26 10:30:12.928 ERROR master/master.go:144 Post "https://127.0.0.1:6443/api/v1/namespaces": dial tcp 127.0.0.1:6443: connect: connection refused
2023-07-26 10:33:01.199 ERROR modules/k3s.go:308 apply /home/kuscia/crds/v1alpha1/kuscia.secretflow_domainappimages.yaml err:exit status 1
2023-07-26 10:33:01.483 ERROR modules/k3s.go:205 exit status 1
2023-07-26 10:33:02.136 ERROR modules/k3s.go:196 context had done, no need to wait to restart
2023-07-26 10:33:03.763 ERROR modules/k3s.go:308 apply /home/kuscia/etc/conf/kuscia-clusterrole.yaml err:exit status 1
2023-07-26 10:33:03.763 ERROR modules/k3s.go:209 exit status 1
2023-07-26 10:33:03.820 ERROR master/master.go:144 Post "https://127.0.0.1:6443/api/v1/namespaces": dial tcp 127.0.0.1:6443: connect: connection refused
2023-07-26 10:36:27.181 ERROR modules/k3s.go:308 apply /home/kuscia/crds/v1alpha1/kuscia.secretflow_interopconfigs.yaml err:exit status 1
2023-07-26 10:36:27.396 ERROR modules/k3s.go:205 exit status 1
2023-07-26 10:36:27.851 ERROR modules/k3s.go:196 context had done, no need to wait to restart
2023-07-26 10:36:30.452 ERROR modules/k3s.go:308 apply /home/kuscia/etc/conf/kuscia-clusterrole.yaml err:exit status 1
2023-07-26 10:36:30.452 ERROR modules/k3s.go:209 exit status 1
2023-07-26 10:36:30.579 ERROR master/master.go:144 Post "https://127.0.0.1:6443/api/v1/namespaces": dial tcp 127.0.0.1:6443: connect: connection refused
2023-07-26 10:37:58.463 ERROR modules/k3s.go:308 apply /home/kuscia/crds/v1alpha1/kuscia.secretflow_taskresourcegroups.yaml err:exit status 1
2023-07-26 10:37:58.691 ERROR modules/k3s.go:205 exit status 1
2023-07-26 10:37:58.964 ERROR modules/k3s.go:196 context had done, no need to wait to restart
2023-07-26 10:37:59.747 ERROR modules/k3s.go:308 apply /home/kuscia/etc/conf/kuscia-clusterrole.yaml err:exit status 1
2023-07-26 10:37:59.747 ERROR modules/k3s.go:209 exit status 1
2023-07-26 10:38:00.014 ERROR master/master.go:144 Post "https://127.0.0.1:6443/api/v1/namespaces": dial tcp 127.0.0.1:6443: connect: connection refused
2023-07-26 10:40:00.953 ERROR modules/k3s.go:308 apply /home/kuscia/etc/conf/kuscia-clusterrolebinding.yaml err:exit status 1
2023-07-26 10:40:01.055 ERROR modules/k3s.go:209 exit status 1
2023-07-26 10:40:01.292 ERROR modules/k3s.go:196 context had done, no need to wait to restart
2023-07-26 10:40:01.468 ERROR master/master.go:144 Post "https://127.0.0.1:6443/api/v1/namespaces": dial tcp 127.0.0.1:6443: connect: connection refused
2023-07-26 10:41:24.850 ERROR modules/kusciaapi.go:141 send health request error: Post "https://127.0.0.1:8082/healthZ": dial tcp 127.0.0.1:8082: connect: connection refused
2023-07-26 10:41:25.241 ERROR modules/kusciaapi.go:141 send health request error: Post "https://127.0.0.1:8082/healthZ": dial tcp 127.0.0.1:8082: connect: connection refused

from kuscia.

@xiaood 辛苦把 k3s.log 文件中的日志也贴一下

from kuscia.

k3s.log

由于这里面内容很长，我做了筛选 grep “error” k3s.log,内容如下：
E0726 10:29:39.201408 57 controller.go:146] failed to ensure lease exists, will retry in 200ms, error: namespaces "kube-system" not found
I0726 10:30:09.192177 57 secure_serving.go:296] http: TLS handshake error from 127.0.0.1:34682: EOF
Trace[1903418969]: ---"Objects listed" error: 12078ms (10:31:02.761)
I0726 10:31:10.685701 56 request.go:1370] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string "json:"apiVersion,omitempty""; Kind string "json:"kind,omitempty"" }
Trace[389143211]: ---"Objects listed" error: 11924ms (10:31:22.765)
Trace[1391844805]: ---"Objects listed" error: 11924ms (10:31:22.766)
Trace[191776647]: ---"Objects listed" error: 10500ms (10:31:22.806)
Trace[1511582215]: ---"Objects listed" error: 10350ms (10:31:22.809)
{"level":"warn","ts":"2023-07-26T10:31:30.868+0800","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc003b60a80/kine.sock","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
[-]etcd failed: error getting data from etcd: context deadline exceeded
I0726 10:31:33.505233 56 request.go:1370] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string "json:"apiVersion,omitempty""; Kind string "json:"kind,omitempty"" }
I0726 10:31:33.505669 56 request.go:1370] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string "json:"apiVersion,omitempty""; Kind string "json:"kind,omitempty"" }
Trace[1274934436]: ---"Objects listed" error: 22363ms (10:31:34.836)
Trace[159336190]: ---"Objects listed" error: 26353ms (10:31:38.813)
time="2023-07-26T10:31:44+08:00" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"
time="2023-07-26T10:33:20+08:00" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"
Trace[626108097]: ---"Objects listed" error: 12367ms (10:33:54.637)
Trace[1574850793]: ---"Objects listed" error: 13117ms (10:33:55.386)
Trace[1156773396]: ---"Objects listed" error: 15558ms (10:34:54.364)
Trace[243085862]: ---"Objects listed" error: 32703ms (10:35:11.509)
Trace[639632167]: ---"Objects listed" error: 48689ms (10:35:27.495)
Trace[309972925]: ---"Objects listed" error: 53034ms (10:35:31.841)
Trace[2055607930]: ---"Objects listed" error: 12855ms (10:35:37.418)
time="2023-07-26T10:36:42+08:00" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"
Trace[1921338498]: ---"Objects listed" error: 9028ms (10:37:53.670)
time="2023-07-26T10:38:12+08:00" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"
time="2023-07-26T10:40:13+08:00" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"

from kuscia.

@xiaood 从k3s.log日志中看，K3s无法正常运行，K3s apiserver 访问storage backend(sqlite) 超时了。麻烦把下面的内容贴一下，我们再分析下。

1. k3s.log日志中的前100行日志(越多越好)
2. 机器的配置信息CPU/Memory/Disk

from kuscia.

2. 机器的配置信息CPU/Memory/Disk

（1）k3s.log日志中的前100行日志：
1 time="2023-07-26T10:29:28+08:00" level=warning msg="Webhooks and apiserver aggregation may not function properly withou 1 t an agent; please set egress-selector-mode to 'cluster' or 'pod'"
2 time="2023-07-26T10:29:28+08:00" level=info msg="Starting k3s v1.26.5+k3s1 (7cefebea)"
3 time="2023-07-26T10:29:28+08:00" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOp 3 enConns=0, connMaxLifetime=0s"
4 time="2023-07-26T10:29:28+08:00" level=info msg="Configuring database table schema and indexes, this may take a moment. 4 .."
5 time="2023-07-26T10:29:28+08:00" level=info msg="Database tables and indexes are up to date"
6 time="2023-07-26T10:29:28+08:00" level=info msg="Kine available at unix://kine.sock"
7 time="2023-07-26T10:29:28+08:00" level=info msg="generated self-signed CA certificate CN=k3s-client-ca@1690338568: notB 7 efore=2023-07-26 02:29:28.414056766 +0000 UTC notAfter=2033-07-23 02:29:28.414056766 +0000 UTC"
8 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:admin,O=system:masters signed by CN=k3s-client-c 8 a@1690338568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
9 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:kube-controller-manager signed by CN=k3s-client- 9 ca@1690338568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
10 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:kube-scheduler signed by CN=k3s-client-ca@169033 10 8568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
11 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:apiserver,O=system:masters signed by CN=k3s-clie 11 nt-ca@1690338568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
12 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1690338568 12 : notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
13 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@169033 13 8568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
14 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=k3s-cloud-controller-manager signed by CN=k3s-client-ca 14 @1690338568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
15 time="2023-07-26T10:29:28+08:00" level=info msg="generated self-signed CA certificate CN=k3s-server-ca@1690338568: notB 15 efore=2023-07-26 02:29:28.42588919 +0000 UTC notAfter=2033-07-23 02:29:28.42588919 +0000 UTC"
16 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-server-ca@1690338568: n
16 otBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
17 time="2023-07-26T10:29:28+08:00" level=info msg="generated self-signed CA certificate CN=k3s-request-header-ca@16903385 17 68: notBefore=2023-07-26 02:29:28.438484038 +0000 UTC notAfter=2033-07-23 02:29:28.438484038 +0000 UTC"
18 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=system:auth-proxy signed by CN=k3s-request-header-ca@16 18 90338568: notBefore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
19 time="2023-07-26T10:29:28+08:00" level=info msg="generated self-signed CA certificate CN=etcd-server-ca@1690338568: not 19 Before=2023-07-26 02:29:28.440639092 +0000 UTC notAfter=2033-07-23 02:29:28.440639092 +0000 UTC"
20 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=etcd-server signed by CN=etcd-server-ca@1690338568: not 20 Before=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
21 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=etcd-client signed by CN=etcd-server-ca@1690338568: not 21 Before=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
22 time="2023-07-26T10:29:28+08:00" level=info msg="generated self-signed CA certificate CN=etcd-peer-ca@1690338568: notBe 22 fore=2023-07-26 02:29:28.443934331 +0000 UTC notAfter=2033-07-23 02:29:28.443934331 +0000 UTC"
23 time="2023-07-26T10:29:28+08:00" level=info msg="certificate CN=etcd-peer signed by CN=etcd-peer-ca@1690338568: notBefo 23 re=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:28 +0000 UTC"
24 time="2023-07-26T10:29:29+08:00" level=info msg="Saving cluster bootstrap data to datastore"
25 time="2023-07-26T10:29:29+08:00" level=info msg="certificate CN=k3s,O=k3s signed by CN=k3s-server-ca@1690338568: notBef 25 ore=2023-07-26 02:29:28 +0000 UTC notAfter=2024-07-25 02:29:29 +0000 UTC"
26 time="2023-07-26T10:29:29+08:00" level=warning msg="dynamiclistener [::]:6443: no cached certificate available for prel 26 oad - deferring certificate load until storage initialization or first client request"
27 time="2023-07-26T10:29:29+08:00" level=info msg="Active TLS secret / (ver=) (count 10): map[listener.cattle.io/cn-10.43 27 .0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.18.0.2:172.18.0.2 listener.cattle.io 27 /cn-__1-f16284:::1 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.defa 27 ult listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cl 27 uster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-wuyish 27 eng-kuscia-master:wuyisheng-kuscia-master listener.cattle.io/fingerprint:SHA1=F9AD35A51A414DA3B27CF728991977514776BDB6] 27 "
28 time="2023-07-26T10:29:29+08:00" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true - 28 -anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC - 28 -bind-address=127.0.0.1 --cert-dir=/home/kuscia/var/k3s/server/tls/temporary-certs --client-ca-file=/home/kuscia/var/k3 28 s/server/tls/client-ca.crt --egress-selector-config-file=/home/kuscia/var/k3s/server/etc/egress-selector-config.yaml -- 28 enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-ser 28 vers=unix://kine.sock --feature-gates=JobTrackingWithFinalizers=true --kubelet-certificate-authority=/home/kuscia/var/k 28 3s/server/tls/server-ca.crt --kubelet-client-certificate=/home/kuscia/var/k3s/server/tls/client-kube-apiserver.crt --ku 28 belet-client-key=/home/kuscia/var/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP 28 ,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/home/kuscia/var/k3s/server/tls/client-auth-proxy.crt - 28 -proxy-client-key-file=/home/kuscia/var/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth- 28 proxy --requestheader-client-ca-file=/home/kuscia/var/k3s/server/tls/request-header-ca.crt --requestheader-extra-header 28 s-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User -- 28 secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/home 28 /kuscia/var/k3s/server/tls/service.key --service-account-signing-key-file=/home/kuscia/var/k3s/server/tls/service.curre 28 nt.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert 28 -file=/home/kuscia/var/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_S 28 HA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA2 28 56,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/home/kuscia/var/ 28 k3s/server/tls/serving-kube-apiserver.key"
29 time="2023-07-26T10:29:29+08:00" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentic 29 ation-kubeconfig=/home/kuscia/var/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/home/kuscia/var/k3s 29 /server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserve 29 r-client-cert-file=/home/kuscia/var/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-fi 29 le=/home/kuscia/var/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/home/kuscia/var/k3s/server 29 /tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/home/kuscia/var/k3s/server/tls/client-ca.key --cl 29 uster-signing-kubelet-serving-cert-file=/home/kuscia/var/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet 29 -serving-key-file=/home/kuscia/var/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/home/kuscia 29 /var/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/home/kuscia/var/k3s/server/tls/ser

（2）机器配置：内存1GB、cpu是1,硬盘40GB

from kuscia.

@xiaood 从k3s.log日志中看，K3s无法正常运行，K3s apiserver 访问storage backend(sqlite) 超时了。麻烦把下面的内容贴一下，我们再分析下。

1. k3s.log日志中的前100行日志(越多越好)
2. 机器的配置信息CPU/Memory/Disk

您说到cpu环境那些，我发现我的配置不符合官网的，感谢，我改一下配置再去试一下

from kuscia.

Hi @xiaood , 请问调整配置后，是否可以正常使用？

from kuscia.

Hi @xiaood , 请问调整配置后，是否可以正常使用？

能正常使用，感谢啦

from kuscia.

centos7系统可以部署吗，请问

from kuscia.

@fly78lv 可以的

from kuscia.