hocus-dev / hocus Goto Github PK

🪄 Spin up ready-to-code, disposable dev environments on your own servers. Self-hosted alternative to Gitpod and Github Codespaces.

Home Page: https://hocus.dev

License: Other

JavaScript 0.47% Shell 6.10% TypeScript 90.50% CSS 0.03% Dockerfile 2.53% PLpgSQL 0.15% Makefile 0.07% Nix 0.15%

hocus's People

Contributors

Stargazers

Watchers

Forkers

stared devdoshi cyberflamego furqanrydhan bhaveshsgupta keyman9848 calsaviour triptych icichainz suryatmodulus kustomzone osamaishtiaq gaurav-chaurasia jrcribb richardsonjf doytsujin victorhaine nanderoo winnerineast albinpraveen grantmacnamara mszczodrak tgz timeowilliams vovanduc gorbak25 archiveproject sadobass duke24k qqq-tech themayurahir ankur612 spread0x bb-drummer surajkamdi medord abdellah-scrimba webdevsharex igecloudsdev gorathon devrutherles victorgomez09 webrulon gmh5225 fairhopeweb henryvonnew stephenxxxx djs55 mrcodechef curiouzk0d3r butangero xops-alebrijecircus-x nemke82 vitran96 monolith-taehyun neuroradiology lhalvesf mister2d paschalcs drasaadmoosa pks-os nebula45644 beingofexistence13 sbarcelona11 ramu-narasinga dearborn-open-ai ninja7239 zhang333jie chilldaddys3 devopstoday11 spaciba joaomlopes

hocus's Issues

🐛 Bug description

Ran the suggested startup command,

git clone https://github.com/hocus-dev/hocus.git
cd hocus
# HOCUS_HOSTNAME is the hostname of the machine where you are deploying Hocus
# If you are running Hocus locally, set it to "localhost"
# If you are running Hocus on a remote server, set it to the hostname of the server
HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh

and got:

hocus-local-hocus-ui-1  | Using 'form_post' for response_mode may cause issues for you logging in over http, see https://github.com/auth0/express-openid-connect/blob/master/FAQ.md
hocus-local-hocus-ui-1  | Express server listening on port 3000
hocus-local-hocus-agent-1  | Trying to load target_core_user
hocus-local-hocus-agent-1  | ./setup-tcmu.sh: line 17: modprobe: command not found

Also, I might add that modprobe is a command on the system when running it manually.

Environment

Hocus version (commit hash)
5c0aa03
OS and its version
Ubuntu Server 22.04.2 LTS x86_64
Any other environment information that may help
Running in a vm in proxmox but using host cpu for kvm acceleration

🐛 Bug description

Followed hocus quick start guide, but got that error than trying to run on my linux machine

namai@parrot ~/hocus (main)
$ HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh
Pulling docker images 📥 - ❌ in 1.23
unknown flag: --ignore-buildable

Above you will find the logs
Please report this problem here
🙏🙏🙏  https://github.com/hocus-dev/hocus/issues/new/choose  🙏🙏🙏

Environment

Hocus version (commit hash) latest ca01333
Parrot OS 5.3 (Electro Ara) Kernel 6.1.0-1parrot1-amd64
Any other environment information that may help

Happy to help to deploy Hocus in Kubernetes env

Hocus is very promising and I'm a kubernetes enthusiast.
If you need some support to port Hocus to k8s don't hesitate to ping me 😄
I am currently trying to do this.

❓ Questions/Help/Support

Hi,

Is it possible to use Hocus with https? I have a domain with HSTS forced (.dev) so http isn't an option for me. I basically have this domain pointed to a machine with caddy on it which terminates the TLS then proxies the request to the hocus server. However, when trying to login after Keycloak redirects back to the main portal, I get "Bad Request" and the logs show:

hocus-local-hocus-ui-1              | BadRequestError: unexpected iss value, expected http://<domain_name>:4200/realms/hocus, got: https://<domain_name>:4200/realms/hocus
hocus-local-hocus-ui-1              |     at /app/node_modules/express-openid-connect/middleware/auth.js:121:19
hocus-local-hocus-ui-1              |     at runMicrotasks (<anonymous>)
hocus-local-hocus-ui-1              |     at processTicksAndRejections (node:internal/process/task_queues:96:5)

Is there a way to have hocus accept the https version of the domain even though the body may not be encrypted?

How to avoid having to log in to services every time?

🚀 Feature

Is there any persisted data between workspaces?
My workflow involves logging into services like Fly.io / Convex, which store auth info in the home directory.
Do I have to log in again every time I start up a development environment, or is there some way to add the auth?

I considered adding it to the config yaml file (to write the auth files), but I wouldn't want to put my login credentials in the file that's checked into the repo.

I'd like to be able to spin up a workspace and immediately be able to run commands that require auth files to already exist in ~/.fly , ~/.convex , etc.

Thanks!

Hocus fails to resolve DNS records hosted locally

🐛 Bug description

When using a self-hosted git and DNS, hocus fails on the prebuild stage for such a repo since it cannot resolve the local name with the hardcoded 8.8.8.8 DNS server.

In my setup, I host a DNS server locally in the network and provide local domain names that get resolved through it. The DNS server IP is given along with the DHCP lease so the host system knows which resolver to use, but Hocus does not inherit this and fails to resolve the name of the local git server.

Steps to reproduce

Install Hocus
Set up a git server in a local network as well as a DNS server with records pointing to that git server
Add a repository to Hocus

Result

The prebuild status for branches stays pending perpetually as it fails to fetch the project since it cannot resolve the local git server with 8.8.8.8.

What should happen

Hocus should be able to resolve local DNS names and pull the repo.

Possible fixes

This issue could be fixed by providing the ability to configure DNS servers used internally instead of hardcoding 8.8.8.8, or by setting up a DNS forwarder that the VMs could use internally that would then forward the requests to the DNS used by the host system.

Environment

Hocus version (commit hash): a998d71
OS and its version: Ubuntu Server 22.04.2

installing on openmediavault

🐛 Bug description

I am trying to install hocus on a old x86 machine, which is currently running openmediavault,
this is the error I am getting, have no idea why:

[setup-vm-images 3/6] RUN apt-get update && apt-get install -y curl wget qemu-system psmisc expect:
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_6.2%2bdfsg-2ubuntu6.8_amd64.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system_6.2%2bdfsg-2ubuntu6.8_amd64.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libg/libglvnd/libglvnd0_1.4.0-1_amd64.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libg/libglvnd/libglx0_1.4.0-1_amd64.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libg/libglvnd/libgl1_1.4.0-1_amd64.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_6.2%2bdfsg-2ubuntu6.8_amd64.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/edk2/ovmf_2022.02-3ubuntu0.22.04.1_all.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/edk2/qemu-efi-aarch64_2022.02-3ubuntu0.22.04.1_all.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/edk2/qemu-efi-arm_2022.02-3ubuntu0.22.04.1_all.deb Could not resolve 'archive.ubuntu.com' [IP: 91.189.91.38 80]
#7 156.7 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Environment

Hocus version (commit hash) : installed through script today
OS and its version: openmediavault 6

Error: unable to list search attributes: Namespace default is not found.

🐛 Bug description

I'm trying to run hocus on my local linux pc.
Temporal seems to be failing to initialize the default namespace.
I'm sure I am missing something trivial, can someone assist?

$ HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh
...
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.619Z","msg":"Started Worker","service":"worker","Namespace":"temporal-system","TaskQueue":"temporal-archival-tq","WorkerID":"1@7b6033569db3@","logging-call-at":"client_worker.go:137"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.620Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-history-scanner-taskqueue-0/1","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.621Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-tq-scanner-taskqueue-0/1","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.625Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-archival-tq/3","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.626Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-archival-tq","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.627Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-archival-tq/2","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.627Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-tq-scanner-taskqueue-0/2","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.627Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-history-scanner-taskqueue-0","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.627Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-archival-tq","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.627Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-history-scanner-taskqueue-0/3","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.639Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-tq-scanner-taskqueue-0","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.639Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-history-scanner-taskqueue-0/2","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.639Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-processor-parent-close-policy/2","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.639Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"7b6033569db3:22688902-a621-4b79-a1f7-119f6715a9b0","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.640Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-processor-parent-close-policy","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.640Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-archival-tq/1","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.641Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-processor-parent-close-policy/1","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.644Z","msg":"Started Worker","service":"worker","Namespace":"temporal-system","TaskQueue":"temporal-sys-processor-parent-close-policy","WorkerID":"1@7b6033569db3@","logging-call-at":"processor.go:98"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.645Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-processor-parent-close-policy","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.646Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-processor-parent-close-policy/3","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.647Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"7b6033569db3:2689b93c-db33-44eb-98d8-bc95ff7c72f0","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.647Z","msg":"Started Worker","service":"worker","Namespace":"temporal-system","TaskQueue":"temporal-sys-batcher-taskqueue","WorkerID":"1@7b6033569db3@","logging-call-at":"batcher.go:98"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.647Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-processor-parent-close-policy/1","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.648Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-batcher-taskqueue/2","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.648Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-processor-parent-close-policy/2","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.648Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-batcher-taskqueue","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.649Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/temporal-sys-batcher-taskqueue/2","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.649Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"7b6033569db3:435c53c3-26b8-48df-ae3c-4ac028718911","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.650Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-batcher-taskqueue","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.650Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/default-worker-tq/1","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.652Z","msg":"Started Worker","service":"worker","Namespace":"temporal-system","TaskQueue":"default-worker-tq","WorkerID":"1@7b6033569db3@","logging-call-at":"worker.go:101"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.652Z","msg":"none","component":"worker-manager","lifecycle":"Started","logging-call-at":"worker.go:106"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.652Z","msg":"none","component":"perns-worker-manager","lifecycle":"Starting","logging-call-at":"pernamespaceworker.go:162"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.652Z","msg":"none","component":"perns-worker-manager","lifecycle":"Started","logging-call-at":"pernamespaceworker.go:173"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.653Z","msg":"worker service started","service":"worker","component":"worker","address":"172.21.0.5:7239","logging-call-at":"service.go:410"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.653Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"/_sys/default-worker-tq/2","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.654Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"default-worker-tq","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.654Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"default-worker-tq","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.655Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"7b6033569db3:b0a6f7d6-5881-44d6-bd57-e8729fe3d277","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.655Z","msg":"Started Worker","service":"worker","Namespace":"temporal-system","TaskQueue":"temporal-sys-per-ns-tq","WorkerID":"server-worker@1@7b6033569db3@temporal-system","logging-call-at":"pernamespaceworker.go:469"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.656Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-per-ns-tq","wf-task-queue-type":"Activity","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:26.656Z","msg":"none","service":"matching","component":"matching-engine","wf-task-queue-name":"temporal-sys-per-ns-tq","wf-task-queue-type":"Workflow","wf-namespace":"temporal-system","lifecycle":"Started","logging-call-at":"taskQueueManager.go:292"}
hocus-local-temporal-1     | + temporal operator cluster health
hocus-local-temporal-1     | + grep -q SERVING
hocus-local-temporal-1     | Temporal server started.
hocus-local-temporal-1     | Registering default namespace: default.
hocus-local-temporal-1     | + echo 'Temporal server started.'
hocus-local-temporal-1     | + [[ false != true ]]
hocus-local-temporal-1     | + register_default_namespace
hocus-local-temporal-1     | + echo 'Registering default namespace: default.'
hocus-local-temporal-1     | + temporal operator namespace describe default
hocus-local-temporal-1     | Error: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | Default namespace default not found. Creating...
hocus-local-temporal-1     | + echo 'Default namespace default not found. Creating...'
hocus-local-temporal-1     | + temporal operator namespace create --retention 1 --description 'Default namespace for Temporal Server.' default
hocus-local-temporal-1     | {"level":"info","ts":"2023-05-24T18:29:27.445Z","msg":"Register namespace succeeded","service":"frontend","wf-namespace":"default","wf-namespace-id":"1f5dcebf-4275-4aeb-9ee7-b51f2ba7e68f","logging-call-at":"namespace_handler.go:309"}
hocus-local-temporal-1     | Namespace default successfully registered.
hocus-local-temporal-1     | Default namespace default registration complete.
hocus-local-temporal-1     | + echo 'Default namespace default registration complete.'
hocus-local-temporal-1     | + [[ false != true ]]
hocus-local-temporal-1     | + add_custom_search_attributes
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + sleep 1
hocus-local-temporal-1     | + temporal operator search-attribute list --namespace default
hocus-local-temporal-1     | Error: unable to list search attributes: Namespace default is not found.
hocus-local-temporal-1     | ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
hocus-local-temporal-1     | Waiting for namespace cache to refresh...
hocus-local-temporal-1     | + echo 'Waiting for namespace cache to refresh...'
hocus-local-temporal-1     | + sleep 1

Above you will find the logs
Please report this problem here
🙏🙏🙏  https://github.com/hocus-dev/hocus/issues/new/choose  🙏🙏🙏
We will get you a 🦝 to help you as quickly as possible

$ docker ps -a
CONTAINER ID   IMAGE                                 COMMAND                  CREATED          STATUS                   PORTS                                                                                                                                                                        NAMES
59b8bbbf3328   temporalio/ui:2.14.0                  "./start-ui-server.sh"   4 minutes ago    Up 4 minutes             0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                                                                                                                                    hocus-local-temporal-ui-1
daff1e353af5   temporalio/admin-tools:1.20.0         "tail -f /dev/null"      4 minutes ago    Up 4 minutes                                                                                                                                                                                          hocus-local-temporal-admin-tools-1
7b6033569db3   temporalio/auto-setup:1.20.0          "/etc/temporal/entry…"   4 minutes ago    Up 4 minutes (healthy)   6933-6935/tcp, 6939/tcp, 7233-7235/tcp, 7239/tcp                                                                                                                             hocus-local-temporal-1
ea959e74e3df   hocus-local-temporal-hocus-codec      "/nodejs/bin/node co…"   4 minutes ago    Created                                                                                                                                                                                               hocus-local-temporal-hocus-codec-1
823b9af31d5f   hocus-local-keycloak                  "/opt/keycloak/bin/k…"   4 minutes ago    Up 4 minutes (healthy)   8443/tcp, 0.0.0.0:4200->8080/tcp, :::4200->8080/tcp                                                                                                                          hocus-local-keycloak-1
5ca161573aef   postgres:15.2-alpine                  "docker-entrypoint.s…"   4 minutes ago    Up 4 minutes (healthy)   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                                                                                                                    hocus-local-db-1

Thank you in advance!

Environment

Hocus version (commit hash) - 10a8fc1
OS and its version - Arch linux, up to date;
Any other environment information that may help

$ docker --version
Docker version 24.0.0, build 98fdcd769b

Error pulling docker images

At some point during startup, docker compose seems to have problems pulling images.

command line output:

➜  hocus git:(main) HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh
Building docker images 👷📦
Building vm-builder done in 1,84 s ✅
Building db-autosetup done in 0,95 s ✅
Building keycloak done in 1,12 s ✅
Building temporal-codec done in 1,93 s ✅
Building ui done in 1,29 s ✅
Building agent done in 1,54 s ✅
Pulling docker images 📥 - ❌ in 0,08

I tried to track down the error which led me to believe that the --ignore-buildable flag might be misplaced (compose says "unknown flag: --ignore-buildable") - if I remove this flag though, I get the following error


➜  hocus git:(main) ✗ HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh
Building docker images 👷📦
Building vm-builder done in 2,13 s ✅
Building db-autosetup done in 0,88 s ✅
Building keycloak done in 1,05 s ✅
Building temporal-codec done in 11,73 s ✅
Building ui done in 8,67 s ✅
Building agent done in 10,64 s ✅
Pulling docker images 📥WARN[0000] The "AGENT_DEV_CREATE_HOCUS_PROJECTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "AGENT_DEV_CREATE_DEVELOPEMENT_PROJECTS" variable is not set. Defaulting to a blank string.
WARN[0000] The "HOCUS_REPO_PRIVATE_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "GITHUB_APP_HOCUS_DEV_CLIENT_SECRET" variable is not set. Defaulting to a blank string.
[+] Running 10/10
 ⠿ hocus-agent Skipped                                                                                                                                                                          0.0s
 ⠿ setup-keycloak Skipped                                                                                                                                                                       0.0s
 ⠿ temporal-hocus-codec Skipped                                                                                                                                                                 0.0s
 ⠿ setup-vm-images Skipped                                                                                                                                                                      0.0s
 ⠿ hocus-ui Skipped                                                                                                                                                                             0.0s
 ⠿ keycloak Skipped                                                                                                                                                                             0.0s
 ⠿ temporal Pulled                                                                                                                                                                              1.2s
 ⠿ db Pulled                                                                                                                                                                                    1.0s
 ⠿ temporal-ui Pulled                                                                                                                                                                           1.1s
 ⠿ temporal-admin-tools Pulled                                                                                                                                                                  1.2s
Pulling docker images 📥 - ✅ in 1,29 s
Building MicroVMs 👷🖥️ - ✅ in 6,43 s
Seeding the DB 🌱 - ✅ in 0,99 s
Starting the DB 📙 - ❌ in 0,10

no such service: setup-keycloak

Above you will find the logs
Please report this problem here
🙏🙏🙏  https://github.com/hocus-dev/hocus/issues/new/choose  🙏🙏🙏
We will get you a 🦝 to help you as quickly as possible

Environment

Hocus version (commit hash): f8dd7db
OS: Ubuntu Jammy LTS 22.2

Podman support

❓ Questions/Help/Support

Hello,
How can I set this up using podman? Will it be ever supported? Thanks.

VSCode extension incorrectly reports SSH agent is not running

🐛 Bug description

I use the 1Password SSH agent locally for handling all my SSH keys. To use the 1Password SSH agent I have to disable and stop the default SSH agent service. When connecting to a Hocus workspace I get a prompt that my SSH agent is not running.

Source:

hocus/extensions/vscode_ui/src/extension.ts

Line 347 in ca01333

// The ssh agent is a global windows service

1Password docs: https://developer.1password.com/docs/ssh/get-started/#step-3-turn-on-the-1password-ssh-agent

It may make more sense to check ssh-add -l on Windows to see if a valid SSH agent is running. Here I run that command with the 1Password SSH agent enabled, and then again after disabling the SSH agent:

PS C:\Users\user> ssh-add -l
256 SHA256:.... SSH Key Name One (ED25519)
256 SHA256:.... SSH Key Name Two (ED25519)
PS C:\Users\user> ssh-add -l
Error connecting to agent: No such file or directory
PS C:\Users\user>

Environment

Hocus extension: v0.0.8

Ubuntu ARM64 Architecture Support

Feature

Requesting the incorporation of Ubuntu ARM64 architecture support into Hocus, optimizing the platform for utilization on affordable ARM-based cloud instances.

Motivation
Driven by the economical nature of ARM systems in the cloud, where instances can be provisioned at a fraction of the cost of traditional x86 instances, this feature aims to tap into the cost-efficiency of ARM64-based hardware. For instance, Oracle offers a complimentary perpetual instance featuring 4 CPUs, 24 GB RAM, and 200 GB storage. Integrating Ubuntu ARM64 support into Hocus would leverage these affordable ARM systems for efficient, budget-friendly Hocus deployments.

Proposal
Incorporate Ubuntu ARM64 architecture compatibility into Hocus, enabling streamlined provisioning, management, and utilization of development environments on ARM64-based cloud instances. Ensure that documentation and user interfaces reflect this new architecture option.

Benefits
Optimized Costs: By capitalizing on ARM64-based instances, Hocus deployments become remarkably cost-effective, appealing to budget-conscious developers and teams.

Enhanced Value: Leveraging ARM64 instances, particularly the free Oracle ARM64 offering, allows users to access substantial computing resources without a financial burden.

Future-Ready: Embracing ARM64 aligns Hocus with the industry's shift towards more affordable and energy-efficient computing solutions.

Alternatives
While community contributions and maintaining a separate ARM64 branch are potential avenues, official integration guarantees compatibility and a seamless user experience on ARM64 systems.

Context
By enabling Ubuntu ARM64 support, Hocus can position itself as an optimal solution for inexpensive yet powerful disposable development environments, catering to developers who seek to maximize their resources while minimizing costs in the cloud computing landscape.

local-up.sh : Error interpreting the docker compose version

🐛 Bug description

Environment

Hocus version (commit hash) : f04d83f
OS : Ubuntu 22.04.3 LTS (jammy)
Any other environment information that may help : Running in VirtualBox (kvm enabled)

Details

Hello, first of all thank you this project and for what has already been achieved. I was looking forward to test the current state of hocus but I encountered a minor error during the first build.

The installation script exited as error because it couldn't interpret the docker compose version, although I installed the latter using the recommended method on the official docker website (DEB file).

I had to do the following change in local-up.sh to go through:

$ git diff
diff --git a/ops/bin/local-up.sh b/ops/bin/local-up.sh
index b2697e2..3ad18b6 100755
--- a/ops/bin/local-up.sh
+++ b/ops/bin/local-up.sh
@@ -170,7 +170,7 @@ if [ -z ${HOCUS_BUILD_COMMIT_HASH+x} ]; then
   EXTRA_PULL_FLAGS=--ignore-buildable
   # Check minimum version of docker compose
   # We require docker compose of at least version 2.15.0 which introduced --ignore-buildable
-  COMPOSE_VERSION="$(docker compose version | grep -o -E '[0-9]+(\.[0-9]+){2}$')"
+  COMPOSE_VERSION="$(docker compose version | grep -o -E '[0-9]+(\.[0-9]+){2}-desktop\.1$')"
   COMPOSE_REQUIRED_VERSION="2.15.0"

For information, here are the brut outputs regarding docker and docker compose versions.

$ docker compose version
Docker Compose version v2.20.2-desktop.1

$ docker version
Client: Docker Engine - Community
 Cloud integration: v1.0.35-desktop+001
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:35:18 2023
 OS/Arch:           linux/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.22.0 (117440)
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:45 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

VScode in browser

Is code server integration will be available later, to use workspace in browser?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

Lock file maintenance

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Update dependency @types/cors to v2.8.13

Detected dependencies

buildkite

ops/buildkite/test.yml

detect-clowns v2.0.0

docker v5.6.0

docker v5.6.0

shellcheck v1.3.0

docker v5.6.0

docker v5.6.0

docker v5.6.0

docker v5.6.0

docker v5.6.0

docker v5.6.0

docker-compose v4.12.0

docker-compose v4.12.0

docker-compose v4.12.0

docker-compose

ops/docker/agent-dev.docker-compose.yml

ops/docker/agent-in-hocus.docker-compose.yml

ops/docker/agent-tests.docker-compose.yml

ops/docker/backend-tests.docker-compose.yml

ops/docker/deploy-smoke-test.yml

ops/docker/docker-compose.yml

ops/docker/hocus-local.yml

postgres 15.2-alpine

temporalio/auto-setup 1.20.0

temporalio/admin-tools 1.20.0

temporalio/ui 2.14.0

ops/docker/temporal.docker-compose.yml

postgres 13-alpine

temporalio/auto-setup 1.20.0

temporalio/admin-tools 1.20.0

temporalio/ui 2.14.0

dockerfile

hocus-workspace/hocus.Dockerfile

gcc 12.2.0

ubuntu 22.04

hocus-workspace/workspace-base.Dockerfile

gcc 12.2.0

ubuntu 22.04

hocus.Dockerfile

ops/docker/agent-tests.Dockerfile

node 18-bullseye

ops/docker/backend-tests.Dockerfile

node 16-bullseye

ops/docker/gitpod.Dockerfile

ops/docker/hocus-agent-dev.Dockerfile

node 18-bullseye

ops/docker/hocus-agent.Dockerfile

node 16-bullseye

node 16-bullseye

node 16-bullseye

ops/docker/hocus-temporal-codec.Dockerfile

node 16-bullseye

ops/docker/hocus-ui.Dockerfile

node 16-bullseye

node 16-bullseye

node 16-bullseye

ops/docker/keycloak-dev.Dockerfile

quay.io/keycloak/keycloak 19.0.2

ops/docker/setup.Dockerfile

resources/docker/base.Dockerfile

ubuntu 22.04

resources/docker/buildfs.Dockerfile

ubuntu 22.04

resources/docker/buildkite-agent.Dockerfile

ubuntu 22.04

resources/docker/checkout-and-inspect.Dockerfile

ubuntu 22.04

resources/docker/default-workspace.Dockerfile

resources/docker/fetchrepo.Dockerfile

ubuntu 22.04

npm

deps/firecracker-client/package.json

@types/node-fetch 2

node-fetch 2

@openapitools/openapi-generator-cli ^2.5.2

extensions/vscode_ui/package.json

fs-extra ^11.1.0

@types/fs-extra ^11.0.1

@types/glob ^8.0.1

@types/mocha ^10.0.1

@types/node 16.x

@types/vscode ^1.75.0

@typescript-eslint/eslint-plugin ^5.49.0

@typescript-eslint/parser ^5.49.0

@vscode/test-electron ^2.2.2

esbuild ^0.17.10

eslint ^8.33.0

glob ^8.1.0

mocha ^10.1.0

typescript ^5.0.0

vscode ^1.75.0

extensions/vscode_workspace/package.json

@sinclair/typebox ^0.25.24

fs-extra ^11.1.0

yaml ^2.2.1

@types/fs-extra ^11.0.1

@types/glob ^8.0.1

@types/mocha ^10.0.1

@types/node 16.x

@types/vscode ^1.75.0

@typescript-eslint/eslint-plugin ^5.49.0

@typescript-eslint/parser ^5.49.0

@vscode/test-electron ^2.2.2

esbuild ^0.17.10

eslint ^8.33.0

glob ^9.2.1

mocha ^10.1.0

ts-toolbelt ^9.6.0

typescript ^5.0.0

vscode ^1.75.0

package.json

@prisma/client 4.7.1

@remix-run/node ^1.11.0

@remix-run/react ^1.11.0

@remix-run/serve ^1.11.0

@sinclair/typebox ^0.25.21

@temporalio/client ^1.5.2

@temporalio/worker ^1.5.2

async-mutex ^0.4.0

axios ^0.27.2

body-parser ^1.20.0

bson ^5.1.0

cbor-x ^1.5.0

cookie-parser ^1.4.6

cors ^2.8.5

csurf ^1.11.0

dockerfile-ast ^0.5.0

dotenv ^16.0.2

express-openid-connect ^2.8.0

flowbite ^1.6.2

flowbite-react 0.4.2

http-status-codes ^2.2.0

jsonwebtoken ^9.0.0

moment ^2.29.4

netmask ^2.0.2

node-ssh ^13.0.0

path-browserify ^1.0.1

prisma 4.7.1

proper-lockfile ^4.1.2

react ^18.2.0

react-dom ^18.2.0

sshpk ^1.17.0

tail ^2.2.4

ts-pattern ^4.1.2

undici ^5.11.0

unique-names-generator ^4.7.1

winston ^3.8.1

yaml ^2.1.3

@remix-run/dev ^1.11.0

@remix-run/eslint-config ^1.11.0

@swc/core ^1.2.241

@swc/helpers ^0.4.7

@swc/jest ^0.2.22

@temporalio/testing ^1.4.4

@types/cookie-parser ^1.4.3

@types/cors 2.8.8

@types/csurf ^1.11.2

@types/express ^4.17.13

@types/jest ^28.1.7

@types/jsonwebtoken ^9.0.1

@types/netmask ^1.0.30

@types/node 16.18

@types/path-browserify ^1.0.0

@types/pg ^8.6.5

@types/proper-lockfile ^4.1.2

@types/react ^18.0.28

@types/react-dom ^18.0.11

@types/ssh2 ^1.11.6

@types/ssh2-streams ^0.1.9

@types/sshpk ^1.17.1

@types/tail ^2.2.1

@types/uuid ^8.3.4

@typescript-eslint/eslint-plugin ^5.33.1

@typescript-eslint/parser ^5.33.1

autoprefixer ^10.4.8

cross-env ^7.0.3

esbuild v0.17.12

eslint ^8.25.0

eslint-plugin-filename-rules ^1.2.0

jest ^28.1.3

nodemon ^2.0.19

npm-run-all ^4.1.5

pg ^8.7.3

postcss ^8.4.16

tailwindcss ^3.1.8

ts-node ^10.9.1

ts-sinon ^2.0.2

ts-toolbelt ^9.6.0

typescript ^5.0.0

uuid ^8.3.2

prettier ^2.8.7

node >=14

Check this box to trigger a request for Renovate to run again on this repository

Failed to boot firecracker VM, failed to fetchRepository?

hocus-local-hocus-agent-1 | 2023-05-07T18:51:16.089Z [INFO] Booting firecracker VM with pid 254 took: 456.09 ms, TOTAL: 505.93 ms
hocus-local-hocus-agent-1 | 2023-05-07T18:51:18.255Z [WARN] firecracker process with pid 254 closed: 0
hocus-local-hocus-agent-1 | 2023-05-07T18:52:11.437Z [WARN] Activity failed {
hocus-local-hocus-agent-1 | error: FetchError: The request failed and the interceptors did not return an alternative response
hocus-local-hocus-agent-1 | at DefaultApi.BaseAPI.fetchApi (/app/node_modules/firecracker-client/dist/runtime.js:100:31)
hocus-local-hocus-agent-1 | at processTicksAndRejections (node:internal/process/task_queues:96:5)
hocus-local-hocus-agent-1 | at DefaultApi.request (/app/node_modules/firecracker-client/dist/runtime.js:136:26)
hocus-local-hocus-agent-1 | ... 5 lines matching cause stack trace ...
hocus-local-hocus-agent-1 | at /app/agent.js:2259:16 {
hocus-local-hocus-agent-1 | cause: TypeError: fetch failed
hocus-local-hocus-agent-1 | at fetch (/app/node_modules/undici/index.js:105:13)
hocus-local-hocus-agent-1 | at processTicksAndRejections (node:internal/process/task_queues:96:5)
hocus-local-hocus-agent-1 | at DefaultApi.BaseAPI.fetchApi (/app/node_modules/firecracker-client/dist/runtime.js:84:28)
hocus-local-hocus-agent-1 | at DefaultApi.request (/app/node_modules/firecracker-client/dist/runtime.js:136:26)
hocus-local-hocus-agent-1 | at DefaultApi.createSyncActionRaw (/app/node_modules/firecracker-client/dist/apis/DefaultApi.js:83:26)
hocus-local-hocus-agent-1 | at DefaultApi.createSyncAction (/app/node_modules/firecracker-client/dist/apis/DefaultApi.js:96:9)
hocus-local-hocus-agent-1 | at FirecrackerService.shutdownVM (/app/agent.js:3817:5)
hocus-local-hocus-agent-1 | at FirecrackerService.withVM (/app/agent.js:3798:9)
hocus-local-hocus-agent-1 | at /app/agent.js:4082:7
hocus-local-hocus-agent-1 | at /app/agent.js:2259:16 {
hocus-local-hocus-agent-1 | cause: [Error]
hocus-local-hocus-agent-1 | }
hocus-local-hocus-agent-1 | },
hocus-local-hocus-agent-1 | durationMs: 55931,
hocus-local-hocus-agent-1 | isLocal: false,
hocus-local-hocus-agent-1 | attempt: 1,
hocus-local-hocus-agent-1 | namespace: 'default',
hocus-local-hocus-agent-1 | taskToken: '...',
hocus-local-hocus-agent-1 | workflowId: '271718a2-c2e1-4e43-8ffc-695c43e5c6d0',
hocus-local-hocus-agent-1 | workflowRunId: '6d4504b2-5b2d-4d8b-89d6-961d0605af71',
hocus-local-hocus-agent-1 | workflowType: 'runBuildfsAndPrebuilds',
hocus-local-hocus-agent-1 | activityId: '2',
hocus-local-hocus-agent-1 | activityType: 'fetchRepository',
hocus-local-hocus-agent-1 | taskQueue: 'main'
hocus-local-hocus-agent-1 | }

Have you tried Cloud Hypervisor?

❓ Questions/Help/Support

I've just read your blogpost about the move from Firecracker to QEMU and the reasons why Firecracker wasn't fitting your needs. Have you ever considered using Cloud Hypervisor? It's been designed for the use cases you describe, and doesn't have any of the limitation that made you move away from Firecracker. And in comparison with QEMU, the codebase is much smaller and written in Rust. It actually relies on multiple Rust-VMM crates which are used from Firecracker as well.

make some setting configurable

🚀 Feature

Running Hocus on my machine returns:
Error response from daemon: driver failed programming external connectivity on endpoint hocus-local-temporal-ui-1 (cbc62ebf21e7d4809eeae9d689060319db484d77827cc32bbb1f2383370580b2): Error starting userland proxy: listen tcp4 0.0.0.0:8080: bind: address already in use

Because I already use a lot of software & reverse-proxies.... So 8080 is already bound.
Would be nice if I could simply change this setting (a.k.a. make it dynamic from a central place)?
Without figuring out what you have build and how you have set it up yourselves.

With Kind Regards,
Ruben Visser

Add the possibility of configuring the VM address range

🚀 Feature Request

Add the possibility of configuring the address range used by VMs internally for situations where the default one might clash with an existing range used by the host system.

Debugging Git access

❓ Questions/Help/Support

I got this message from VS Code after connecting to the workspace: SSH keys were forwarded but we were unable to access the git repo!

Is there anyway to debug this issue? The key is forwarded correctly as I can manually fetch/push with a terminal. It seems that the StrictHostChecking no option is not present in the host config, thus the connection "probably" fails due to The authenticity of host 'github.com ()' can't be established. error, but I have no way to confirm this.

Ran local-up.sh with appropriate HOCUS_HOSTNAME and the log appeared

🐛 Bug description

I ran /ops/bin/local-up.sh and the message Above you will find the vm build logs with the errors highlighted appeared.

Above you will find the vm build logs with the errors highlighted
Please report this problem here
���  https://github.com/hocus-dev/hocus/issues/new/choose  ���
We will get you a � to help you as quickly as possible

Running this on a PhotonOS 4 VM with virtualization enabled on ESXi 6.7.0 Update 3 (Build 15160138)

I actually don't see any error or issue but maybe I'm missing it? The last messages say there was an error and it definitely did not start.

Environment

Hocus version (commit hash): main branch, commit ff7bab1
PhotonOS 4 (Running on vmware ESXi 6.7.0 Update 3 (Build 15160138) with hardware virtualization passthrough enabled
Screenshot attached for more info

Log attached below

hocus.log.txt

Cannot install on ubuntu 22.10

latest version: 6dc0c21

Server from DigitalOcean

root@lazerg:/var/www/hocus# HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh
Building docker images 👷📦
Building db-autosetup done in 2.13 s ✅
Building keycloak done in 1.43 s ✅
Building temporal-codec done in 2.22 s ✅
Building ui done in 1.96 s ✅
Building agent done in 279.81 s ✅
Pulling docker images 📥 - ✅ in 37.14 s
Seeding the DB 🌱 - ✅ in 2.46 s
Starting the DB 📙 - ✅ in 8.73 s
Starting Keycloak 🔑 - ✅ in 32.67 s
Starting Temporal ☁️  - ✅ in 13.68 s
Starting Hocus 🧙🪄  - ❌ in 29.53

hocus-local-hocus-ui-1  | Prisma schema loaded from prisma/schema.prisma
hocus-local-hocus-ui-1  | Datasource "db": PostgreSQL database "rooms", schema "public" at "db:5432"
hocus-local-hocus-ui-1  | 
hocus-local-hocus-ui-1  | 
hocus-local-hocus-ui-1  | 38 migrations found in prisma/migrations
hocus-local-hocus-ui-1  | 
hocus-local-hocus-ui-1  | 
hocus-local-hocus-ui-1  | Applying migration `20220925182738_uuid`
hocus-local-hocus-ui-1  | Applying migration `20220925182923_user`
hocus-local-hocus-ui-1  | Applying migration `20221215151914_logs`
hocus-local-hocus-ui-1  | Applying migration `20221219184401_git`
hocus-local-hocus-ui-1  | Applying migration `20221220152544_git_branch`
hocus-local-hocus-ui-1  | Applying migration `20230103150633_project`
hocus-local-hocus-ui-1     | Applying migration `20230107154927_project_external_id`
hocus-local-hocus-ui-1     | Applying migration `20230112143235_workspace`
hocus-local-hocus-ui-1     | Applying migration `20230115194912_prebuild_status`
hocus-local-hocus-ui-1     | Applying migration `20230116180502_workspace_instance_index`
hocus-local-hocus-ui-1     | Applying migration `20230221135444_git_branch_external_id`
hocus-local-hocus-ui-1     | Applying migration `20230222101358_workspace_id_unique`
hocus-local-hocus-ui-1     | Applying migration `20230302150058_vm_task_external_id`
hocus-local-hocus-ui-1     | Applying migration `20230306155237_user_env_var_set`
hocus-local-hocus-ui-1     | Applying migration `20230306160822_env_var_external_id`
hocus-local-hocus-ui-1     | Applying migration `20230308092813_user_ssh_key`
hocus-local-hocus-ui-1     | Applying migration `20230310163538_i_hate_bash_too_much`
hocus-local-hocus-ui-1     | Applying migration `20230311214214_workspace_delete`
hocus-local-hocus-ui-1     | Applying migration `20230314182046_prebuild`
hocus-local-hocus-ui-1     | Applying migration `20230316180624_archive_prebuild_event`
hocus-local-hocus-ui-1     | Applying migration `20230320121907_prebuild_monitoring_workflow`
hocus-local-hocus-ui-1     | Applying migration `20230320172445_prebuild_pending_archive`
hocus-local-hocus-ui-1     | Applying migration `20230321120251_user_git_config`
hocus-local-hocus-ui-1     | Applying migration `20230322194347_user_active`
hocus-local-hocus-ui-1     | Applying migration `20230331104136_project_drive_ram_vcpu`
hocus-local-hocus-ui-1     | Applying migration `20230331115218_rm_prebuild_project_size`
hocus-local-hocus-ui-1     | Applying migration `20230412105459_repo_connection_status`
hocus-local-hocus-ui-1     | Applying migration `20230414140124_prebuild_system_error`
hocus-local-hocus-ui-1     | Applying migration `20230424131123_workspace_stopped_with_error`
hocus-local-hocus-ui-1     | Applying migration `20230425135113_git_object_to_branch`
hocus-local-hocus-ui-1     | Applying migration `20230427105252_prebuild_archive_after`
hocus-local-hocus-ui-1     | Applying migration `20230523142016_prebuild_event_workflow_id`
hocus-local-hocus-ui-1     | Applying migration `20230607174735_local_oci_image`
hocus-local-hocus-ui-1     | Applying migration `20230612142826_file_to_image`
hocus-local-hocus-ui-1     | Applying migration `20230613110250_buildfs_project_image`
hocus-local-hocus-ui-1     | Applying migration `20230615134541_workspace_instance`
hocus-local-hocus-ui-1     | Applying migration `20230622160500_prebuild_workspace_drive_size`
hocus-local-hocus-ui-1     | Applying migration `20230706101524_buildfs_external_id`
hocus-local-hocus-ui-1     | 
hocus-local-hocus-ui-1     | 
hocus-local-hocus-ui-1     | The following migrations have been applied:
hocus-local-hocus-ui-1     | 
hocus-local-hocus-ui-1     | 
hocus-local-hocus-ui-1     | migrations/
hocus-local-hocus-ui-1     |   └─ 20220925182738_uuid/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20220925182923_user/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20221215151914_logs/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20221219184401_git/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20221220152544_git_branch/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230103150633_project/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230107154927_project_external_id/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230112143235_workspace/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230115194912_prebuild_status/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230116180502_workspace_instance_index/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-agent-1  | Trying to load target_core_user
hocus-local-hocus-agent-1  | Trying to load tcm_loop
hocus-local-hocus-agent-1  | SCSI scan mode not in sync mode, enabling sync mode
hocus-local-hocus-agent-1  | Starting OpenBSD Secure Shell server: sshd.
hocus-local-hocus-agent-1  | net.ipv4.conf.vpeer-vms.proxy_arp = 1
hocus-local-hocus-agent-1  | net.ipv6.conf.vpeer-vms.disable_ipv6 = 1
hocus-local-hocus-agent-1  | net.ipv4.conf.vpeer-ssh-vms.proxy_arp = 1
hocus-local-hocus-agent-1  | net.ipv6.conf.vpeer-ssh-vms.disable_ipv6 = 1
hocus-local-hocus-agent-1  | net.ipv4.conf.veth-vms.proxy_arp = 1
hocus-local-hocus-agent-1  | net.ipv6.conf.veth-vms.disable_ipv6 = 1
hocus-local-hocus-agent-1  | 2023-07-18T09:58:55.163Z [INFO] Found TCMU: Target Engine Core ConfigFS Infrastructure v5.0 on Linux/x86_64 on 5.19.0-23-generic
hocus-local-hocus-agent-1  | 
hocus-local-hocus-agent-1  | 
hocus-local-hocus-agent-1  | 2023-07-18T09:58:55.384Z [INFO] Found TCM LOOP: TCM Loopback Fabric module v2.1-rc2
hocus-local-hocus-agent-1  | 
hocus-local-hocus-agent-1  | 
hocus-local-hocus-agent-1  | Not setting up any projects
hocus-local-hocus-agent-1  | 2023-07-18T09:58:56.320Z [INFO] Creating worker {
hocus-local-hocus-agent-1  |   options: {
hocus-local-hocus-agent-1  |     namespace: 'default',
hocus-local-hocus-agent-1  |     identity: '1@fb3c2480ef72',
hocus-local-hocus-agent-1  |     shutdownGraceTime: 0,
hocus-local-hocus-agent-1  |     maxConcurrentActivityTaskExecutions: 100,
hocus-local-hocus-agent-1  |     maxConcurrentLocalActivityExecutions: 100,
hocus-local-hocus-agent-1  |     enableNonLocalActivities: true,
hocus-local-hocus-agent-1  |     maxConcurrentWorkflowTaskExecutions: 100,
hocus-local-hocus-agent-1  |     stickyQueueScheduleToStartTimeout: '10s',
hocus-local-hocus-agent-1  |     maxHeartbeatThrottleInterval: '60s',
hocus-local-hocus-agent-1  |     defaultHeartbeatThrottleInterval: '30s',
hocus-local-hocus-agent-1  |     isolateExecutionTimeout: '5s',
hocus-local-hocus-agent-1  |     workflowThreadPoolSize: 8,
hocus-local-hocus-agent-1  |     maxCachedWorkflows: 250,
hocus-local-hocus-agent-1  |     enableSDKTracing: false,
hocus-local-hocus-agent-1  |     showStackTraceSources: false,
hocus-local-hocus-agent-1  |     reuseV8Context: false,
hocus-local-hocus-agent-1  |     debugMode: false,
hocus-local-hocus-agent-1  |     interceptors: { activityInbound: [Array], workflowModules: [Array] },
hocus-local-hocus-agent-1  |     sinks: { defaultWorkerLogger: [Object] },
hocus-local-hocus-agent-1  |     connection: NativeConnection {
hocus-local-hocus-agent-1  |       nativeClient: [External: 4ef8c30],
hocus-local-hocus-agent-1  |       referenceHolders: Set(0) {}
hocus-local-hocus-agent-1  |     },
hocus-local-hocus-agent-1  |     workflowBundle: { codePath: '/app/workflow-bundle.js' },
hocus-local-hocus-agent-1  |     activities: {
hocus-local-hocus-agent-1  |       fetchRepository: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       buildfs: [Function (anonymous)],
hocus-local-hocus-agent-1  |       checkoutAndInspect: [Function (anonymous)],
hocus-local-hocus-agent-1  |       prebuild: [Function (anonymous)],
hocus-local-hocus-ui-1     |   └─ 20230221135444_git_branch_external_id/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230222101358_workspace_id_unique/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230302150058_vm_task_external_id/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230306155237_user_env_var_set/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230306160822_env_var_external_id/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230308092813_user_ssh_key/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230310163538_i_hate_bash_too_much/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230311214214_workspace_delete/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230314182046_prebuild/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230316180624_archive_prebuild_event/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230320121907_prebuild_monitoring_workflow/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230320172445_prebuild_pending_archive/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230321120251_user_git_config/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230322194347_user_active/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230331104136_project_drive_ram_vcpu/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230331115218_rm_prebuild_project_size/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230412105459_repo_connection_status/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230414140124_prebuild_system_error/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230424131123_workspace_stopped_with_error/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230425135113_git_object_to_branch/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230427105252_prebuild_archive_after/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230523142016_prebuild_event_workflow_id/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230607174735_local_oci_image/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230612142826_file_to_image/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230613110250_buildfs_project_image/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230615134541_workspace_instance/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230622160500_prebuild_workspace_drive_size/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |   └─ 20230706101524_buildfs_external_id/
hocus-local-hocus-ui-1     |     └─ migration.sql
hocus-local-hocus-ui-1     |       
hocus-local-hocus-ui-1     | All migrations have been successfully applied.
hocus-local-hocus-ui-1     | npm notice 
hocus-local-hocus-ui-1     | npm notice New major version of npm available! 8.19.4 -> 9.8.0
hocus-local-hocus-ui-1     | npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.8.0>
hocus-local-hocus-ui-1     | npm notice Run `npm install -g [email protected]` to update!
hocus-local-hocus-ui-1     | npm notice 
hocus-local-hocus-ui-1     | Using 'form_post' for response_mode may cause issues for you logging in over http, see https://github.com/auth0/express-openid-connect/blob/master/FAQ.md
hocus-local-hocus-ui-1     | Express server listening on port 3000
hocus-local-hocus-agent-1  |       changePrebuildEventStatus: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       createWorkspace: [Function (anonymous)],
hocus-local-hocus-agent-1  |       startWorkspace: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       stopWorkspace: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getOrCreateBuildfsEvents: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       createPrebuildImages: [Function (anonymous)],
hocus-local-hocus-agent-1  |       getWorkspaceInstanceStatus: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       addProjectAndRepository: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getRepositoryProjects: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       updateGitBranchesAndObjects: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getDefaultBranch: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       deleteWorkspace: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getOrCreatePrebuildEvents: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       initPrebuildEvents: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getPrebuildEvents: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       reservePrebuildEvent: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       removePrebuildEventReservation: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       waitForPrebuildEventReservations: [Function (anonymous)],
hocus-local-hocus-agent-1  |       markPrebuildEventAsArchived: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       deleteLocalPrebuildEventFiles: [Function (anonymous)],
hocus-local-hocus-agent-1  |       deleteRemovablePrebuildEvents: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getArchivablePrebuildEvents: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       saveGitRepoConnectionStatus: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       cleanUpAfterPrebuildError: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       cleanUpWorkspaceInstanceLocal: [Function (anonymous)],
hocus-local-hocus-agent-1  |       signalWithStartLockWorkflow: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getWorkflowStatus: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       cleanUpWorkspaceInstanceDb: [Function (anonymous)],
hocus-local-hocus-agent-1  |       createPrebuildEvent: [Function (anonymous)],
hocus-local-hocus-agent-1  |       signalWithStartWaitWorkflow: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       getProjectsRepository: [AsyncFunction (anonymous)],
hocus-local-hocus-agent-1  |       removeContentWithPrefix: [Function (anonymous)]
hocus-local-hocus-agent-1  |     },
hocus-local-hocus-agent-1  |     taskQueue: 'main',
hocus-local-hocus-agent-1  |     dataConverter: { payloadConverterPath: '/app/data-converter.js' },
hocus-local-hocus-agent-1  |     shutdownGraceTimeMs: 0,
hocus-local-hocus-agent-1  |     shutdownForceTimeMs: undefined,
hocus-local-hocus-agent-1  |     stickyQueueScheduleToStartTimeoutMs: 10000,
hocus-local-hocus-agent-1  |     isolateExecutionTimeoutMs: 5000,
hocus-local-hocus-agent-1  |     maxHeartbeatThrottleIntervalMs: 60000,
hocus-local-hocus-agent-1  |     defaultHeartbeatThrottleIntervalMs: 30000,
hocus-local-hocus-agent-1  |     loadedDataConverter: {
hocus-local-hocus-agent-1  |       payloadConverter: [CompositePayloadConverter2],
hocus-local-hocus-agent-1  |       failureConverter: [DefaultFailureConverter],
hocus-local-hocus-agent-1  |       payloadCodecs: []
hocus-local-hocus-agent-1  |     }
hocus-local-hocus-agent-1  |   }
hocus-local-hocus-agent-1  | }
hocus-local-hocus-agent-1  | 2023-07-18T09:59:07.975865Z  INFO temporal_sdk_core::worker: Initializing worker task_queue=main namespace=default
hocus-local-hocus-agent-1  | Starting worker...
hocus-local-hocus-agent-1  | 2023-07-18T09:59:07.988Z [INFO] Worker state changed { state: 'RUNNING' }
 Container hocus-local-hocus-agent-1  Creating
 Container hocus-local-hocus-ui-1  Creating
 Container hocus-local-hocus-agent-1  Created
 Container hocus-local-hocus-ui-1  Created
 Container hocus-local-hocus-agent-1  Starting
 Container hocus-local-hocus-ui-1  Starting
 Container hocus-local-hocus-agent-1  Started
 Container hocus-local-hocus-ui-1  Started
 Container hocus-local-hocus-ui-1  Waiting
 Container hocus-local-hocus-agent-1  Waiting
 Container hocus-local-hocus-ui-1  Healthy
container hocus-local-hocus-agent-1 is unhealthy

Above you will find the logs
Please report this problem here
🙏🙏🙏  https://github.com/hocus-dev/hocus/issues/new/choose  🙏🙏🙏

Evaluate Profile-Guided Optimization (PGO) and LLVM BOLT on QEMU

Hi! Thanks a lot for the project!

Recently I read a great article about "Firecracker vs QEMU" in your blog - link. Since you are using QEMU and (hopefully) you care about the QEMU performance, I have an idea of how to improve QEMU performance for your scenarios.

Some time ago, I started evaluating Profile-Guided Optimization (PGO) improvements on different projects in different domains. Current results are available here. According to the tests (not only mine), PGO can help with achieving better performance. The most interesting stuff is that there are the benchmarks for QEMU (link) and CrosVM (link) about improvements in these projects with PGO.

I can suggest the following possible steps:

Perform the PGO benchmarks for QEMU in your scenarios.
Add a note about improvements in Hocus's performance with PGO-optimized QEMU somewhere in the documentation. Even a few percent can be important to someone (especially if Hocus cares about performance).
Providing an easier way (e.g. a build option) to build scripts with PGO can be useful for the end-users and maintainers since they will be able to optimize Hocus according to their own workloads. I understand that QEMU is not your project but maybe you have a way to work with the upstream - who knows... :)

If you already did some benchmarks about PGO-optimized QEMU for your platform - would be great if you could share your numbers.

Thanks in advance!

Hocus Agent stuck on restarting because "/run/netns/vms": File exists

🐛 Bug description

The Hocus Agent docker container looks like it is stuck on restarting. I don't know why or how. This is probably why I can't create a project.

sudo docker ps says

CONTAINER ID   IMAGE                           COMMAND                  CREATED        STATUS                          PORTS
                          NAMES
d6592b3c7030   hocus-local-hocus-agent         "docker-entrypoint.s…"   42 hours ago   Restarting (1) 26 seconds ago
                          hocus-local-hocus-agent-1
7ca69898d314   hocus-local-hocus-ui            "docker-entrypoint.s…"   42 hours ago   Up 42 minutes (healthy)         0.0.0.0:3000->3000/tcp, :::3000->3000/tcp             hocus-local-hocus-ui-1
c2f9644557e6   hocus-local-keycloak            "/opt/keycloak/bin/k…"   42 hours ago   Up 42 minutes (healthy)         8443/tcp, 0.0.0.0:4200->8080/tcp, :::4200->8080/tcp   hocus-local-keycloak-1
d726836c7758   temporalio/auto-setup:1.20.0    "/etc/temporal/entry…"   42 hours ago   Up 42 minutes (healthy)         6933-6935/tcp, 6939/tcp, 7233-7235/tcp, 7239/tcp      hocus-local-temporal-1
888680bf811c   postgres:15.2-alpine            "docker-entrypoint.s…"   42 hours ago   Up 42 minutes (healthy)         0.0.0.0:5432->5432/tcp, :::5432->5432/tcp             hocus-local-db-1
2fc3ebf54eae   cloudflare/cloudflared:latest   "cloudflared --no-au…"   42 hours ago   Up 42 minutes
                          cloudflare

sudo docker logs -f hocus-local-hocus-agent-1 says

Starting OpenBSD Secure Shell server: sshd.
net.ipv4.conf.vpeer-vms.proxy_arp = 1
net.ipv6.conf.vpeer-vms.disable_ipv6 = 1
net.ipv4.conf.vpeer-ssh-vms.proxy_arp = 1
net.ipv6.conf.vpeer-ssh-vms.disable_ipv6 = 1
net.ipv4.conf.veth-vms.proxy_arp = 1
net.ipv6.conf.veth-vms.disable_ipv6 = 1
2023-09-28T17:54:41.748Z [INFO] Found TCMU: Target Engine Core ConfigFS Infrastructure v5.0 on Linux/x86_64 on 5.15.0-84-generic

2023-09-28T17:54:41.750Z [INFO] Found TCM LOOP: TCM Loopback Fabric module v2.1-rc2

Not setting up any projects
2023-09-28T17:54:41.783Z [INFO] Creating worker {
  options: {
    namespace: 'default',
    identity: '1@d6592b3c7030',
    shutdownGraceTime: 0,
    maxConcurrentActivityTaskExecutions: 100,
    maxConcurrentLocalActivityExecutions: 100,
    enableNonLocalActivities: true,
    maxConcurrentWorkflowTaskExecutions: 100,
    stickyQueueScheduleToStartTimeout: '10s',
    maxHeartbeatThrottleInterval: '60s',
    defaultHeartbeatThrottleInterval: '30s',
    isolateExecutionTimeout: '5s',
    workflowThreadPoolSize: 8,
    maxCachedWorkflows: 261,
    enableSDKTracing: false,
    showStackTraceSources: false,
    reuseV8Context: false,
    debugMode: false,
    interceptors: { activityInbound: [Array], workflowModules: [Array] },
    sinks: { defaultWorkerLogger: [Object] },
    connection: NativeConnection {
      nativeClient: [External: 5ccede0],
      referenceHolders: Set(0) {}
    },
    workflowBundle: { codePath: '/app/workflow-bundle.js' },
    activities: {
      fetchRepository: [AsyncFunction (anonymous)],
      buildfs: [Function (anonymous)],
      checkoutAndInspect: [Function (anonymous)],
      prebuild: [Function (anonymous)],
      changePrebuildEventStatus: [AsyncFunction (anonymous)],
      createWorkspace: [Function (anonymous)],
      startWorkspace: [AsyncFunction (anonymous)],
      stopWorkspace: [AsyncFunction (anonymous)],
      getOrCreateBuildfsEvents: [AsyncFunction (anonymous)],
      createPrebuildImages: [Function (anonymous)],
      getWorkspaceInstanceStatus: [AsyncFunction (anonymous)],
      addProjectAndRepository: [AsyncFunction (anonymous)],
      getRepositoryProjects: [AsyncFunction (anonymous)],
      updateGitBranchesAndObjects: [AsyncFunction (anonymous)],
      getDefaultBranch: [AsyncFunction (anonymous)],
      deleteWorkspace: [AsyncFunction (anonymous)],
      getOrCreatePrebuildEvents: [AsyncFunction (anonymous)],
      initPrebuildEvents: [AsyncFunction (anonymous)],
      getPrebuildEvents: [AsyncFunction (anonymous)],
      reservePrebuildEvent: [AsyncFunction (anonymous)],
      removePrebuildEventReservation: [AsyncFunction (anonymous)],
      waitForPrebuildEventReservations: [Function (anonymous)],
      markPrebuildEventAsArchived: [AsyncFunction (anonymous)],
      deleteLocalPrebuildEventFiles: [Function (anonymous)],
      deleteRemovablePrebuildEvents: [AsyncFunction (anonymous)],
      getArchivablePrebuildEvents: [AsyncFunction (anonymous)],
      saveGitRepoConnectionStatus: [AsyncFunction (anonymous)],
      cleanUpAfterPrebuildError: [AsyncFunction (anonymous)],
      cleanUpWorkspaceInstanceLocal: [Function (anonymous)],
      signalWithStartLockWorkflow: [AsyncFunction (anonymous)],
      getWorkflowStatus: [AsyncFunction (anonymous)],
      cleanUpWorkspaceInstanceDb: [Function (anonymous)],
      createPrebuildEvent: [Function (anonymous)],
      signalWithStartWaitWorkflow: [AsyncFunction (anonymous)],
      getProjectsRepository: [AsyncFunction (anonymous)],
      removeContentWithPrefix: [Function (anonymous)]
    },
    taskQueue: 'main',
    dataConverter: { payloadConverterPath: '/app/data-converter.js' },
    shutdownGraceTimeMs: 0,
    shutdownForceTimeMs: undefined,
    stickyQueueScheduleToStartTimeoutMs: 10000,
    isolateExecutionTimeoutMs: 5000,
    maxHeartbeatThrottleIntervalMs: 60000,
    defaultHeartbeatThrottleIntervalMs: 30000,
    loadedDataConverter: {
      payloadConverter: [CompositePayloadConverter2],
      failureConverter: [DefaultFailureConverter],
      payloadCodecs: []
    }
  }
}
2023-09-28T17:54:42.530510Z  INFO temporal_sdk_core::worker: Initializing worker task_queue=main namespace=default
Starting worker...
2023-09-28T17:54:42.531Z [INFO] Worker state changed { state: 'RUNNING' }
Trying to load kernel module target_core_user
Trying to load kernel module tcm_loop
SCSI scan mode not in sync mode, enabling sync mode
Cannot create namespace file "/run/netns/vms": File exists
Cannot create namespace file "/run/netns/vms": File exists
# the last line continues for like another 50 times.

Environment

Hocus version (commit hash) git log says f04d83f9592a0cf719701b6a9e990b5e59616c9c

OS and its version
OS is an Ubuntu VM inside Hyper V running on Windows 11 Pro (Version: 21H2) (Build: 22000.2416)

uname -a says Linux ubuntu-server 5.15.0-84-generic #93-Ubuntu SMP Tue Sep 5 17:16:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/os-release says

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Any other environment information that may help
sudo docker --version says Docker version 20.10.24, build 297e128

Great team, congrats on the launch 🔥

Error during install: ops/bin/buildfs.sh: line 28: docker: command not found

When I run the local-up.sh script I get an error that Docker cannot be found. It looks like Docker is missing in the agent image:

➜ hocus git:(main) HOCUS_HOSTNAME="localhost" ops/bin/local-up.sh

Building docker images 👷📦
Building vm-builder done in 0.66 s ✅
Building db-autosetup done in 0.70 s ✅
Building keycloak done in 0.84 s ✅
Building temporal-codec done in 1.50 s ✅
Building ui done in 1.14 s ✅
Building agent done in 1.12 s ✅
Pulling docker images 📥 - ✅ in 2.37 s
Building MicroVMs 👷🖥️ - ❌ in 0.80
time="2023-07-08T11:16:45-10:00" level=warning msg="The \"AGENT_DEV_CREATE_HOCUS_PROJECTS\" variable is not set. Defaulting to a blank string."
time="2023-07-08T11:16:45-10:00" level=warning msg="The \"AGENT_DEV_CREATE_DEVELOPMENT_PROJECTS\" variable is not set. Defaulting to a blank string."
time="2023-07-08T11:16:45-10:00" level=warning msg="The \"HOCUS_REPO_PRIVATE_KEY\" variable is not set. Defaulting to a blank string."
time="2023-07-08T11:16:45-10:00" level=warning msg="The \"GITHUB_APP_HOCUS_DEV_CLIENT_SECRET\" variable is not set. Defaulting to a blank string."
+ mkdir -pv /agent_data/firecracker
+ mkdir -pv /agent_data/resources
+ ops/bin/buildfs.sh resources/docker/checkout-and-inspect.Dockerfile /agent_data/resources/checkout-and-inspect.ext4 resources/ 500
+ '[' -z resources/docker/checkout-and-inspect.Dockerfile ']'
+ '[' -z /agent_data/resources/checkout-and-inspect.ext4 ']'
+ '[' -z resources/ ']'
+ '[' -z 500 ']'
+ set -o nounset
++ realpath resources/docker/checkout-and-inspect.Dockerfile
+ DOCKERFILE_PATH=/build/resources/docker/checkout-and-inspect.Dockerfile
++ realpath /agent_data/resources/checkout-and-inspect.ext4
+ OUTPUT_PATH=/agent_data/resources/checkout-and-inspect.ext4
++ realpath resources/
+ CONTEXT_DIR=/build/resources
+ FS_MAX_SIZE_MIB=500
++ basename /build/resources/docker/checkout-and-inspect.Dockerfile
++ sed 's/\.Dockerfile//'
+ IMAGE_TAG=checkout-and-inspect
+ IMAGE_NAME=buildfs:checkout-and-inspect
++ cat /dev/urandom
++ tr -dc a-zA-Z0-9
++ head -c 8
++ true
+ CONTAINER_TAG=vzPUJNJ0
+ CONTAINER_NAME=container-buildfs-vzPUJNJ0
+ MOUNT_PATH=/tmp/buildfs-checkout-and-inspect
++ pwd
+ ORIGINAL_PATH=/build
+ docker build --progress=plain --tag buildfs:checkout-and-inspect --file /build/resources/docker/checkout-and-inspect.Dockerfile /build/resources
ops/bin/buildfs.sh: line 28: docker: command not found

Environment

Hocus version: 5ca804d
Host OS is Ubuntu 22.04 with latest updates

Docker is installed on the host, but I see this error is coming from the agent container with is Ubuntu 20. I added the following to buildfs.sh for additional debug output:

cat /etc/os-release
echo $PATH
ls -al /usr/bin | grep docker

Which yields:

+ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
+ echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ ls -al /usr/bin
+ grep docker

Integration with Nix development environments

🚀 Feature

Nix has primarily remained a third-class citizen of most of these types of systems (like the authors, I've tried many times to make Gitpod work with little success). I've always thought these systems were a prime candidate for integration with Nix's development shells. We've already done some work with getting VSCode to support remote development with native Nix containers. Using firecracker seems more promising as we can likely get a more native experience and avoid some of the limitations we ran into with containers.

Unfortunately, I wasn't able to discern enough from the documentation for me to understand if this is something that can currently be supported with Hocus. Are we able to control Hocus at the base image level? Could we hack into the pipeline to build and deploy custom Nix images with development shells pre-populated?

I'd be happy to work on this piece of the integration if it seems plausible.

Service discovery mechanism for hocus workspace

Implementing a service discovery mechanism similar to the one used by Kubernetes would be very helpful. This would allow workspaces to communicate with each other using specific service names, making it easy to integrate with the continuous integration and deployment mechanisms on the Kubernetes platform.

The service discovery mechanism will enable users to easily discover and connect to services between their workspaces using the exposed service name. This approach is also friendly to micro-service development.

Feature: Custom base image with CUDA

🚀 Feature

Would like to use this for machine learning development, but the base image would need to be modified to include CUDA. Are there any plans or workarounds for this?

hocus-dev / hocus Goto Github PK

hocus's People

Contributors

Stargazers

Watchers

Forkers

hocus's Issues

🐛 Bug description

Environment

🐛 Bug description

Environment

❓ Questions/Help/Support

🚀 Feature

🐛 Bug description

Steps to reproduce

Result

What should happen

Possible fixes

Environment

🐛 Bug description

Environment

🐛 Bug description

Environment

Environment

❓ Questions/Help/Support

🐛 Bug description

Environment

Feature

🐛 Bug description

Environment

Details

Awaiting Schedule

Rate-Limited

Open

Ignored or Blocked

Detected dependencies

❓ Questions/Help/Support

🚀 Feature

🚀 Feature Request

❓ Questions/Help/Support

🐛 Bug description

Environment

🐛 Bug description

Environment

Error during install: ops/bin/buildfs.sh: line 28: docker: command not found

Environment

🚀 Feature

🚀 Feature

Recommend Projects

Recommend Topics

Recommend Org