Comments (19)
Cryptophobia
commented on June 11, 2024
From @dumyan on June 14, 2017 16:38
The push is rejected because of the Internal Server Error that happened probably when pushing the built image into registry (I'm not sure if a successful push to registry is logged). This is desired behavior because if the refs get pushed and the image cannot be successfully started in a container it will leave you with a bogus repository state which is not currently deployed. So, failed to push some refs is really your friend here and your enemies are probably hiding in the logs.
In short, you should inspect deis-builder and also deis-registry logs and work your way from there. You can also enable debug mode if you see nothing obvious in the logs.
from workflow.
Cryptophobia
commented on June 11, 2024
From @Overdrivr on June 15, 2017 6:1
Thanks for the reply. The registry pod if filled with info messages, is this the default behavior ?
10.12.0.1 - - [15/Jun/2017:05:53:12 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "Go-http-client/1.1"
time="2017-06-15T05:53:12.050694806Z" level=info msg="response completed" go.version=go1.7.3 http.request.host="10.11.0.12:5000" http.request.id=fe682b3f-5756-47d9-8e86-cf19f5d1a9e3 http.request.method=GET http.request.remoteaddr="10.12.0.1:35296" http.request.uri="/v2/" http.request.useragent="Go-http-client/1.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=8.927239ms http.response.status=200 http.response.written=2 instance.id=dd03c229-fbfe-4309-95c3-3667e7847fe1 service=registry version=v2.6.0
10.12.0.1 - - [15/Jun/2017:05:53:12 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "Go-http-client/1.1"
time="2017-06-15T05:53:22.041218766Z" level=info msg="response completed" go.version=go1.7.3 http.request.host="10.11.0.12:5000" http.request.id=710364ac-dde4-4bb2-9723-cd2d383ca686 http.request.method=GET http.request.remoteaddr="10.12.0.1:35330" http.request.uri="/v2/" http.request.useragent="Go-http-client/1.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.648287ms http.response.status=200 http.response.written=2 instance.id=dd03c229-fbfe-4309-95c3-3667e7847fe1 service=registry version=v2.6.0
10.12.0.1 - - [15/Jun/2017:05:53:22 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "Go-http-client/1.1"
time="2017-06-15T05:53:22.04325269Z" level=info msg="response completed" go.version=go1.7.3 http.request.host="10.11.0.12:5000" http.request.id=80e1cbd3-52e5-405c-ab1c-928957f731de http.request.method=GET http.request.remoteaddr="10.12.0.1:35328" http.request.uri="/v2/" http.request.useragent="Go-http-client/1.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.529342ms http.response.status=200 http.response.written=2 instance.id=dd03c229-fbfe-4309-95c3-3667e7847fe1 service=registry version=v2.6.0
The deis-builder log is maybe more interesting
$ kubectl --namespace=deis logs deis-builder-1134541883-4mvsv
2017/06/14 04:37:38 Starting health check server on port 8092
2017/06/14 04:37:38 Starting deleted app cleaner
2017/06/14 04:37:38 Starting SSH server on 0.0.0.0:2223
Listening on 0.0.0.0:2223
Accepting new connections.
---> ---> 2017/06/14 04:37:51 Readinesscheck endpoint timed out after 10s
2017/06/14 04:38:01 Readinesscheck endpoint timed out after 10s
2017/06/14 04:38:11 Readinesscheck endpoint timed out after 10s
2017/06/14 04:38:21 Readinesscheck endpoint timed out after 10s
2017/06/14 04:38:31 Readinesscheck endpoint timed out after 10s
2017/06/14 04:38:41 Readinesscheck endpoint timed out after 10s
2017/06/14 04:38:51 Readinesscheck endpoint timed out after 10s
2017/06/14 04:39:01 Readinesscheck endpoint timed out after 10s
Accepted connection.
Starting ssh authentication
Channel type: session
receiving git repo name: app-dispatcher.git, operation: git-receive-pack, fingerprint: 84:a8:7e:6a:e8:1b:20:34:4b:29:a7:f3:14:5a:e3:42, user: gitlabci
creating repo directory /home/git/app-dispatcher.git
writing pre-receive hook under /home/git/app-dispatcher.git
git-shell -c git-receive-pack 'app-dispatcher.git'
Waiting for git-receive to run.
Waiting for deploy.
Deploy complete.
Accepted connection.
Starting ssh authentication
Channel type: session
receiving git repo name: app-dispatcher.git, operation: git-receive-pack, fingerprint: 84:a8:7e:6a:e8:1b:20:34:4b:29:a7:f3:14:5a:e3:42, user: gitlabci
creating repo directory /home/git/app-dispatcher.git
writing pre-receive hook under /home/git/app-dispatcher.git
git-shell -c git-receive-pack 'app-dispatcher.git'
Waiting for git-receive to run.
Waiting for deploy.
Deploy complete.
Accepted connection.
---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> ---> [ERROR] Failed handshake: EOF
Accepted connection.
Accepted connection.
---> ---> [ERROR] Failed handshake: EOF
[ERROR] Failed handshake: EOF
Accepted connection.
Accepted connection.
---> ---> [ERROR] Failed handshake: EOF
Accepted connection.
---> [ERROR] Failed handshake: EOF
Any idea what is causing the handshake to fail ?
from workflow.
Cryptophobia
commented on June 11, 2024
From @bacongobbler on June 15, 2017 6:2
This handshake failures are just the healthcheck probe. It's a red herring. What you want to be looking at is the registry logs as that's where the 500 error came from.
from workflow.
Cryptophobia
commented on June 11, 2024
From @Overdrivr on June 15, 2017 6:30
Here you go, registry logs with HTTP 500 error
$ kubectl --namespace=deis logs deis-registry-2623437609-x1fcp | grep " 500 "
10.12.0.7 - - [15/Jun/2017:06:15:15 +0000] "PUT /v2/app-dispatcher/blobs/uploads/a0a37674-7680-4e64-a809-05551e124f76?_state=9rUDYtxw_nxrKtVQVQK20EXSMhdleDG4Ci5BbZEZOjt7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiYTBhMzc2NzQtNzY4MC00ZTY0LWE4MDktMDU1NTFlMTI0Zjc2IiwiT2Zmc2V0Ijo0MzIyNTU2NywiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNDozOFoifQ%3D%3D&digest=sha256%3A4f38f9d5c3c0e3e4eabe6d4dac4bcc729caf847c172d0af75caf581b6c23120c HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:15:20 +0000] "PUT /v2/app-dispatcher/blobs/uploads/05e4cd44-be93-459c-99a4-cf7d77f6519a?_state=omTy6YEGZqDnBs2fqLPOXwWbg6PTtpBzGAh8eVtsIMh7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiMDVlNGNkNDQtYmU5My00NTljLTk5YTQtY2Y3ZDc3ZjY1MTlhIiwiT2Zmc2V0Ijo1MjU2ODI2OSwiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNDozOVoifQ%3D%3D&digest=sha256%3Aef0380f84d05d3cdc5a5f660232d35ccb020ccf1d635b585580dea44691a13a7 HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:15:54 +0000] "PUT /v2/app-dispatcher/blobs/uploads/f1cb1743-43c3-478b-a4b8-27c4ad3e4a8b?_state=xhjHl05OxmV09G4oskfU0CIC7gsbaczC5Evz1iX36jx7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiZjFjYjE3NDMtNDNjMy00NzhiLWE0YjgtMjdjNGFkM2U0YThiIiwiT2Zmc2V0Ijo0MzIyNTU2NywiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNToyMVoifQ%3D%3D&digest=sha256%3A4f38f9d5c3c0e3e4eabe6d4dac4bcc729caf847c172d0af75caf581b6c23120c HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:15:55 +0000] "PUT /v2/app-dispatcher/blobs/uploads/d3fbfd1a-eb57-4cf8-a55b-fb26e0d38ac6?_state=qd0QZx4wanTPwiDZ7aVnXpr1vxR7fMZ_gHTEdmC8b1N7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiZDNmYmZkMWEtZWI1Ny00Y2Y4LWE1NWItZmIyNmUwZDM4YWM2IiwiT2Zmc2V0Ijo1MjU2ODI2OSwiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNToyNloifQ%3D%3D&digest=sha256%3Aef0380f84d05d3cdc5a5f660232d35ccb020ccf1d635b585580dea44691a13a7 HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:15:56 +0000] "PUT /v2/app-dispatcher/blobs/uploads/2a7e85ff-768e-4407-b2e2-810261e0e5d7?_state=2mDRWSFFneGaJL4styC3IQ7Cd1bjnpw5FcOUZpiLUNB7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiMmE3ZTg1ZmYtNzY4ZS00NDA3LWIyZTItODEwMjYxZTBlNWQ3IiwiT2Zmc2V0IjoxMzE4MTkyMDcsIlN0YXJ0ZWRBdCI6IjIwMTctMDYtMTVUMDY6MTQ6MzhaIn0%3D&digest=sha256%3A4125326b53d8463d437a9052d8fb892e1bf9e7dc8d68518ac9780611476123ec HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:16:47 +0000] "PUT /v2/app-dispatcher/blobs/uploads/4213afc5-bd0c-4e89-8545-c7c62d97c973?_state=33N3ehczUMDazWB9sxMH7wN8ftYcLOL4hWyps1L_3P17Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiNDIxM2FmYzUtYmQwYy00ZTg5LTg1NDUtYzdjNjJkOTdjOTczIiwiT2Zmc2V0Ijo0MzIyNTU2NywiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNjowNVoifQ%3D%3D&digest=sha256%3A4f38f9d5c3c0e3e4eabe6d4dac4bcc729caf847c172d0af75caf581b6c23120c HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:16:48 +0000] "PUT /v2/app-dispatcher/blobs/uploads/2670a319-80f6-4b31-9c60-8498fb2219c0?_state=wAgGDjU4XQc-K3tBivJ11HF0bi2rgxXNaocIxf5T9pB7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiMjY3MGEzMTktODBmNi00YjMxLTljNjAtODQ5OGZiMjIxOWMwIiwiT2Zmc2V0Ijo1MjU2ODI2OSwiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNjowNloifQ%3D%3D&digest=sha256%3Aef0380f84d05d3cdc5a5f660232d35ccb020ccf1d635b585580dea44691a13a7 HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:16:55 +0000] "PUT /v2/app-dispatcher/blobs/uploads/46b17962-9a71-4e45-ae03-23b207184548?_state=aunuxweYQROhndJe4ZqAsmnCBLyWbq6r1P-DKY1p-nR7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiNDZiMTc5NjItOWE3MS00ZTQ1LWFlMDMtMjNiMjA3MTg0NTQ4IiwiT2Zmc2V0IjoxMzE4MTkyMDcsIlN0YXJ0ZWRBdCI6IjIwMTctMDYtMTVUMDY6MTY6MDJaIn0%3D&digest=sha256%3A4125326b53d8463d437a9052d8fb892e1bf9e7dc8d68518ac9780611476123ec HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:17:36 +0000] "PUT /v2/app-dispatcher/blobs/uploads/3a644e41-d3c2-44a8-9596-ba42359aa71c?_state=1WfuVKBc2gc3qjb0siH7Lcbf2asHkHjdBDjYOh8T_ZB7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiM2E2NDRlNDEtZDNjMi00NGE4LTk1OTYtYmE0MjM1OWFhNzFjIiwiT2Zmc2V0Ijo0MzIyNTU2NywiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNzowMloifQ%3D%3D&digest=sha256%3A4f38f9d5c3c0e3e4eabe6d4dac4bcc729caf847c172d0af75caf581b6c23120c HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:17:44 +0000] "PUT /v2/app-dispatcher/blobs/uploads/787c7538-1a4e-439c-ad51-a0658ef87e74?_state=b7y6Wvq1uz0pKW1PluUWMABckM3NtIRP6X9zu0IghcZ7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiNzg3Yzc1MzgtMWE0ZS00MzljLWFkNTEtYTA2NThlZjg3ZTc0IiwiT2Zmc2V0Ijo1MjU2ODI2OSwiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNzowNFoifQ%3D%3D&digest=sha256%3Aef0380f84d05d3cdc5a5f660232d35ccb020ccf1d635b585580dea44691a13a7 HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:17:55 +0000] "PUT /v2/app-dispatcher/blobs/uploads/4b5c16e3-2c1e-4264-a6f4-8cc396affbac?_state=vM16auRY8mezk2pPhMIHKLIzgzVhjV43KRhxQihhYid7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiNGI1YzE2ZTMtMmMxZS00MjY0LWE2ZjQtOGNjMzk2YWZmYmFjIiwiT2Zmc2V0IjoxMzE4MTkyMDcsIlN0YXJ0ZWRBdCI6IjIwMTctMDYtMTVUMDY6MTc6MDVaIn0%3D&digest=sha256%3A4125326b53d8463d437a9052d8fb892e1bf9e7dc8d68518ac9780611476123ec HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:18:14 +0000] "PUT /v2/app-dispatcher/blobs/uploads/b3928bcf-36e5-4000-886c-7288fe3389c0?_state=Uhwuv4nSaQCI_MUn8Eeqc8q8OzO1KM74-_YwRZ7JER57Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiYjM5MjhiY2YtMzZlNS00MDAwLTg4NmMtNzI4OGZlMzM4OWMwIiwiT2Zmc2V0Ijo0MzIyNTU2NywiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxNzo1N1oifQ%3D%3D&digest=sha256%3A4f38f9d5c3c0e3e4eabe6d4dac4bcc729caf847c172d0af75caf581b6c23120c HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:18:38 +0000] "PUT /v2/app-dispatcher/blobs/uploads/106c9553-86b6-4dfa-8b22-14bbf712b80f?_state=-5FSF76b4zMHCcWh2qKOfujnGVMTYZYp8dC7c73EFuB7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiMTA2Yzk1NTMtODZiNi00ZGZhLThiMjItMTRiYmY3MTJiODBmIiwiT2Zmc2V0Ijo1MjU2ODI2OSwiU3RhcnRlZEF0IjoiMjAxNy0wNi0xNVQwNjoxODowNVoifQ%3D%3D&digest=sha256%3Aef0380f84d05d3cdc5a5f660232d35ccb020ccf1d635b585580dea44691a13a7 HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:18:49 +0000] "PUT /v2/app-dispatcher/blobs/uploads/f5c80612-d1a4-44b7-9f1a-75a6096718ed?_state=BS_ZRX1Sv0kdnvr25YbNzOG8-f7jjJef2aTRlFKWf317Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiZjVjODA2MTItZDFhNC00NGI3LTlmMWEtNzVhNjA5NjcxOGVkIiwiT2Zmc2V0IjoxMzE4MTkyMDcsIlN0YXJ0ZWRBdCI6IjIwMTctMDYtMTVUMDY6MTg6MTBaIn0%3D&digest=sha256%3A4125326b53d8463d437a9052d8fb892e1bf9e7dc8d68518ac9780611476123ec HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
10.12.0.7 - - [15/Jun/2017:06:19:38 +0000] "PUT /v2/app-dispatcher/blobs/uploads/0592b1a6-5234-458d-9c8a-fa324a050517?_state=iNYDPE62ScgvCj825K8D-xthNqd_YffLVV2JfTzoQRd7Ik5hbWUiOiJjcm9ub2JvLWRpc3BhdGNoZXIiLCJVVUlEIjoiMDU5MmIxYTYtNTIzNC00NThkLTljOGEtZmEzMjRhMDUwNTE3IiwiT2Zmc2V0IjoxMzE4MTkyMDcsIlN0YXJ0ZWRBdCI6IjIwMTctMDYtMTVUMDY6MTk6MDlaIn0%3D&digest=sha256%3A4125326b53d8463d437a9052d8fb892e1bf9e7dc8d68518ac9780611476123ec HTTP/1.0" 500 117 "" "docker/1.11.2 go/go1.7.4 git-commit/4dc5990 kernel/4.4.35+ os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
from workflow.
Cryptophobia
commented on June 11, 2024
From @bacongobbler on June 15, 2017 6:37
Sounds like it's having a hard time uploading images to the backing storage driver as it's barking during PUT operations. I'd check your object storage logs and see if it's crashing, or perhaps your cluster is misconfigured and the registry is unable to upload blobs to object storage.
from workflow.
Cryptophobia
commented on June 11, 2024
From @Overdrivr on June 15, 2017 6:51
$ kubectl --namespace=deis logs deis-minio-3370481340-8nlfx
minio server /home/minio/
2017/06/14 04:37:02 starting Minio server
2017/06/14 04:37:02 starting health check server on 0.0.0.0:8082
Migration from version ‘2’ to ‘3’ completed successfully.
Migration from version ‘3’ to ‘4’ completed successfully.
AccessKey: **************** SecretKey: ******************** Region: us-east-1
Minio Object Storage:
http://127.0.0.1:9000
http://**.**.**.**:9000
Minio Browser:
http://127.0.0.1:9000
http://**.**.**.**:9000
To configure Minio Client:
$ wget https://dl.minio.io/client/mc/release/linux-amd64/mc
$ chmod 755 mc
$ ./mc config host add myminio http://127.0.0.1:9000 **************** *************************
It's weird that minio is using region us-east-1, because my cluster is located in europe-west1-b. Could this be the problem ?
from workflow.
Cryptophobia
commented on June 11, 2024
From @bacongobbler on June 15, 2017 14:11
Minio fakes out and simulates that it's in us-east-1 to mimic the S3 API. It doesn't accurately reflect where it's deployed (in the cluster).
from workflow.
Cryptophobia
commented on June 11, 2024
From @EamonZhang on July 4, 2017 7:45
+1
from workflow.
Cryptophobia
commented on June 11, 2024
From @wikkid on August 7, 2017 15:12
Any updates on this? Got exactly the same problem.
I have bare metal kubernetes cluster and Deis workflow on top of that.
Sample applications are fine, but it doesn't work with my own.
from workflow.
Cryptophobia
commented on June 11, 2024
From @robeferre on August 8, 2017 16:55
Hi Guys any update on this Im having the same issue
from workflow.
Cryptophobia
commented on June 11, 2024
From @Overdrivr on August 9, 2017 4:25
I wasn't sure how to debug this further, so no updates so far. I wasn't seeing anything in my object storage logs.
from workflow.
Cryptophobia
commented on June 11, 2024
From @galbacarys on August 18, 2017 16:45
Having watched this carefully (I know, great way to debug things) I noticed something interesting-It looks like some of the minio requests are going through, but then the connection gets dropped somewhere (the object is too big?) and minio barfs and stops taking any more data.
[ deis registry logs ]
100.96.6.18 - - [18/Aug/2017:16:40:25 +0000] "POST /v2/aws-dx-selfjoin/blobs/uploads/ HTTP/1.0" 202 0 "" "docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
time="2017-08-18T16:40:27.09674817Z" level=info msg="response completed" go.version=go1.7.3 http.request.host="100.96.6.23:5000" http.request.id=cd874d81-d36b-48ec-8455-4528504f88f8 http.request.method=GET http.request.remoteaddr="100.96.6.1:53204" http.request.uri="/v2/" http.request.useragent="Go-http-client/1.1" h
ttp.response.contenttype="application/json; charset=utf-8" http.response.duration=1.746968ms http.response.status=200 http.response.written=2 instance.id=bdee4072-d5d4-4f37-b206-f5555479cc40 service=registry version=v2.6.0
100.96.6.1 - - [18/Aug/2017:16:40:27 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "Go-http-client/1.1"
time="2017-08-18T16:40:27.09757101Z" level=info msg="response completed" go.version=go1.7.3 http.request.host="100.96.6.23:5000" http.request.id=72f0a983-963a-499e-89d6-d30c1b29fab7 http.request.method=GET http.request.remoteaddr="100.96.6.1:53202" http.request.uri="/v2/" http.request.useragent="Go-http-client/1.1" h
ttp.response.contenttype="application/json; charset=utf-8" http.response.duration=1.58666ms http.response.status=200 http.response.written=2 instance.id=bdee4072-d5d4-4f37-b206-f5555479cc40 service=registry version=v2.6.0
100.96.6.1 - - [18/Aug/2017:16:40:27 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "Go-http-client/1.1"
time="2017-08-18T16:40:35.222805315Z" level=info msg="response completed" go.version=go1.7.3 http.request.host="127.0.0.1:5555" http.request.id=6aaf4e8a-dc20-4dcb-a9da-9f2df76db945 http.request.method=PATCH http.request.remoteaddr=100.96.6.1 http.request.uri="/v2/aws-dx-selfjoin/blobs/uploads/dfbcfd73-21a3-43bd-b51b-
9ab733dd225f?_state=0CLtxmCoH9cYWnDAEmjo9mWBMC3_Q7TX85-MY80Au4B7Ik5hbWUiOiJhd3MtZHgtc2VsZmpvaW4iLCJVVUlEIjoiZGZiY2ZkNzMtMjFhMy00M2JkLWI1MWItOWFiNzMzZGQyMjVmIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE3LTA4LTE4VDE2OjQwOjI1LjIxODg4NzY5MVoifQ%3D%3D" http.request.useragent="docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4
.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)" http.response.duration=1.380391587s http.response.status=202 http.response.written=0 instance.id=bdee4072-d5d4-4f37-b206-f5555479cc40 service=registry version=v2.6.0
100.96.6.18 - - [18/Aug/2017:16:40:33 +0000] "PATCH /v2/aws-dx-selfjoin/blobs/uploads/dfbcfd73-21a3-43bd-b51b-9ab733dd225f?_state=0CLtxmCoH9cYWnDAEmjo9mWBMC3_Q7TX85-MY80Au4B7Ik5hbWUiOiJhd3MtZHgtc2VsZmpvaW4iLCJVVUlEIjoiZGZiY2ZkNzMtMjFhMy00M2JkLWI1MWItOWFiNzMzZGQyMjVmIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE3LTA4LTE4VDE2
OjQwOjI1LjIxODg4NzY5MVoifQ%3D%3D HTTP/1.0" 202 0 "" "docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
time="2017-08-18T16:40:35.378123268Z" level=error msg="unknown error completing upload: s3aws: NoSuchKey: The specified key does not exist.\n\tstatus code: 404, request id: DZFQC2IC58RHYLEY" go.version=go1.7.3 http.request.host="127.0.0.1:5555" http.request.id=c3edb847-d84f-4394-b247-0d03d594ce4f http.request.method=
PUT http.request.remoteaddr=100.96.6.1 http.request.uri="/v2/aws-dx-selfjoin/blobs/uploads/dfbcfd73-21a3-43bd-b51b-9ab733dd225f?_state=V01hMgbvQsi-hTA76R9ljuV8B8En0ZGQNiIOy4i6d8N7Ik5hbWUiOiJhd3MtZHgtc2VsZmpvaW4iLCJVVUlEIjoiZGZiY2ZkNzMtMjFhMy00M2JkLWI1MWItOWFiNzMzZGQyMjVmIiwiT2Zmc2V0Ijo0NjIxMDUwOSwiU3RhcnRlZEF0IjoiMjA
xNy0wOC0xOFQxNjo0MDoyNVoifQ%3D%3D&digest=sha256%3A2df4c3c3bdb9ad0afa98766c03c7546f930bec5e854b80e312e7b3bdd0907845" http.request.useragent="docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)" instance.id=bdee4072-d5d4-4f37-b206-f5555479cc40 service=regis
try vars.name=aws-dx-selfjoin vars.uuid=dfbcfd73-21a3-43bd-b51b-9ab733dd225f version=v2.6.0
time="2017-08-18T16:40:35.378179784Z" level=error msg="error canceling upload after error: already closed" go.version=go1.7.3 http.request.host="127.0.0.1:5555" http.request.id=c3edb847-d84f-4394-b247-0d03d594ce4f http.request.method=PUT http.request.remoteaddr=100.96.6.1 http.request.uri="/v2/aws-dx-selfjoin/blobs/u
ploads/dfbcfd73-21a3-43bd-b51b-9ab733dd225f?_state=V01hMgbvQsi-hTA76R9ljuV8B8En0ZGQNiIOy4i6d8N7Ik5hbWUiOiJhd3MtZHgtc2VsZmpvaW4iLCJVVUlEIjoiZGZiY2ZkNzMtMjFhMy00M2JkLWI1MWItOWFiNzMzZGQyMjVmIiwiT2Zmc2V0Ijo0NjIxMDUwOSwiU3RhcnRlZEF0IjoiMjAxNy0wOC0xOFQxNjo0MDoyNVoifQ%3D%3D&digest=sha256%3A2df4c3c3bdb9ad0afa98766c03c7546f93
0bec5e854b80e312e7b3bdd0907845" http.request.useragent="docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)" instance.id=bdee4072-d5d4-4f37-b206-f5555479cc40 service=registry vars.name=aws-dx-selfjoin vars.uuid=dfbcfd73-21a3-43bd-b51b-9ab733dd225f version
=v2.6.0
time="2017-08-18T16:40:35.383839588Z" level=error msg="response completed with error" err.code=unknown err.detail="s3aws: NoSuchKey: The specified key does not exist.\n\tstatus code: 404, request id: DZFQC2IC58RHYLEY" err.message="unknown error" go.version=go1.7.3 http.request.host="127.0.0.1:5555" http.request.id=c3
edb847-d84f-4394-b247-0d03d594ce4f http.request.method=PUT http.request.remoteaddr=100.96.6.1 http.request.uri="/v2/aws-dx-selfjoin/blobs/uploads/dfbcfd73-21a3-43bd-b51b-9ab733dd225f?_state=V01hMgbvQsi-hTA76R9ljuV8B8En0ZGQNiIOy4i6d8N7Ik5hbWUiOiJhd3MtZHgtc2VsZmpvaW4iLCJVVUlEIjoiZGZiY2ZkNzMtMjFhMy00M2JkLWI1MWItOWFiNzMz
ZGQyMjVmIiwiT2Zmc2V0Ijo0NjIxMDUwOSwiU3RhcnRlZEF0IjoiMjAxNy0wOC0xOFQxNjo0MDoyNVoifQ%3D%3D&digest=sha256%3A2df4c3c3bdb9ad0afa98766c03c7546f930bec5e854b80e312e7b3bdd0907845" http.request.useragent="docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)" http.re
sponse.contenttype="application/json; charset=utf-8" http.response.duration=110.023913ms http.response.status=500 http.response.written=117 instance.id=bdee4072-d5d4-4f37-b206-f5555479cc40 service=registry vars.name=aws-dx-selfjoin vars.uuid=dfbcfd73-21a3-43bd-b51b-9ab733dd225f version=v2.6.0
100.96.6.18 - - [18/Aug/2017:16:40:35 +0000] "PUT /v2/aws-dx-selfjoin/blobs/uploads/dfbcfd73-21a3-43bd-b51b-9ab733dd225f?_state=V01hMgbvQsi-hTA76R9ljuV8B8En0ZGQNiIOy4i6d8N7Ik5hbWUiOiJhd3MtZHgtc2VsZmpvaW4iLCJVVUlEIjoiZGZiY2ZkNzMtMjFhMy00M2JkLWI1MWItOWFiNzMzZGQyMjVmIiwiT2Zmc2V0Ijo0NjIxMDUwOSwiU3RhcnRlZEF0IjoiMjAxNy0wOC
0xOFQxNjo0MDoyNVoifQ%3D%3D&digest=sha256%3A2df4c3c3bdb9ad0afa98766c03c7546f930bec5e854b80e312e7b3bdd0907845 HTTP/1.0" 500 117 "" "docker/1.12.6 go/go1.6.4 git-commit/78d1802 kernel/4.4.65-k8s os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)"
from workflow.
Cryptophobia
commented on June 11, 2024
From @dumyan on August 18, 2017 21:59
Hey, guys, you can try with a newer minio image - just edit the minio deployment. Rereading this made me vaguely remember that I had some issues with the minio version that deis deploys by default.
from workflow.
Cryptophobia
commented on June 11, 2024
From @battlemidget on August 19, 2017 20:16
@dumyan What new minio image are you referring to?
from workflow.
Cryptophobia
commented on June 11, 2024
From @dumyan on August 20, 2017 12:10
@battlemidget the OP is using minio as a storage backend for the registry. The default image for deis-minio is pretty outdated (or at least was outdated few months ago when I used minio as a backend storage). My suggestion is to change the minio image in the deis-minio deployment (you can try this image for example minio/minio:RELEASE.2017-07-24T18-27-35Z)
from workflow.
Cryptophobia
commented on June 11, 2024
From @battlemidget on August 20, 2017 20:21
@dumyan thanks im hitting this same issue, ill give that a try
from workflow.
Cryptophobia
commented on June 11, 2024
From @battlemidget on August 21, 2017 17:27
@dumyan Ok, so not exactly the same issue as this one it seems. I am using off cluster storage (s3) and our distribution of kubernetes uses flannel cni so I just needed to enable that.
FYI, we are heavily promoting Deis workflow as part of our core addons for our Canonical Distribution of Kubernetes, my first blog post on it is here http://blog.astokes.org/conjure-up-dev-summary-aws-cloud-native-integration-and-vsphere-3/#addons, and the application is https://conjure-up.io. cc @bacongobbler
Thanks for the help!
from workflow.
Cryptophobia
commented on June 11, 2024
From @bacongobbler on August 21, 2017 18:52
@battlemidget you may want to read our latest blog post. https://deis.com/blog/2017/deis-workflow-final-release/
from workflow.
Cryptophobia
commented on June 11, 2024
From @battlemidget on August 21, 2017 18:58
@bacongobbler well that is disappointing :(
from workflow.
Related Issues (20)
- AWS ALB/NLB support?
- Docs: broken link(s) to Deis blog HOT 7
- ssh_exchange_identification: read: Connection reset by peer HOT 7
- There is an incredibly amount of verbose output when a buildpack is not matched HOT 5
- Helm install fails HOT 2
- Simple installation of Hephy wouldn't work. Client on Mac OS X and Cluster on AWS HOT 13
- Unable to install hephy workflow HOT 18
- How to manage application databases with workflow? HOT 6
- `deis certs` doesn't show certs added by another (admin) user HOT 2
- Unable to install onto Kubernetes 1.18.3 HOT 2
- ps:restart should rolling restart the pods HOT 2
- deis logs is not returning pod logs HOT 7
- Guide for Rancher HOT 6
- error upgrading from very old install, hephy/postgres:v2.7.3 boots but hephy/postgres:v2.7.6 does not. HOT 5
- Chart is broken. HOT 1
- [feature request] Enable to drop monitor HOT 1
- [feature support] Enable to add affinities. HOT 6
- Is this still actively maintained? HOT 2
- Automated release of workflow/docs HOT 2
- Add x-forwarded-host to the nginx config in the deis-router for ruby apps issue HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from workflow.