temporalio / docker-builds Goto Github PK

View Code? Open in Web Editor NEW

22.0 22.0 47.0 852 KB

Temporal service Docker images build

Home Page: https://hub.docker.com/r/temporaliotest/auto-setup

License: MIT License

Dockerfile 12.19% Makefile 15.02% Shell 45.71% Go 20.89% HCL 6.18%

docker-builds's People

Contributors

Stargazers

Watchers

Forkers

crt-fork yolken-airplane art-technologies dexterorion pedronet4k cosminnechifor p2yh wisdom-lin instaclustr badideasforsale github-banc91 aybabtme nipunaprashan osmnnunl osmnnunal gtoonstra mrk-97 emitamago penguintoast michaelsnowden axiom-security mjameswh janbuecker xybstone hardiksethi22 aboutphilippe igsaf nansky hiramhuang mlinebaughppa inkinworld zappy-shu hieesulee elb3k nscloud-demo dandavison bergundy mruoss ego stephanos thisisareku gerardvivancos bchikirivao viktor-kollegov wafi-inc andrewloux

docker-builds's Issues

Where has the docker image 1.22.2 gone?

Hey guys, what happened to docker image 1.22.2? And what is 1.22.0.0 => What version scheme are you guys using?

You knoq, if you publish a container image on docker hub, people (and renovate bot) are going to pick up on that. But now that image is gone and setups are broken...

How to check health?

Suppose I start auto-start container

at 19:08:39 ❯ docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS          PORTS                                                                      NAMES
a5851582cf65   temporalio/admin-tools:1.20.2   "tail -f /dev/null"      12 minutes ago   Up 12 minutes                                                                              temporal-admin-tools
43ae871d5abb   temporalio/web:1.15.0           "docker-entrypoint.s…"   12 minutes ago   Up 12 minutes   0.0.0.0:8088->8088/tcp                                                     temporal-web
18e2280b8d3c   temporalio/auto-setup:1.20.2    "/etc/temporal/entry…"   12 minutes ago   Up 12 minutes   6933-6935/tcp, 6939/tcp, 7234-7235/tcp, 7239/tcp, 0.0.0.0:7233->7233/tcp   temporal

I would expect to be able to health check using:

docker exec -it 18e2280b8d3c temporal operator cluster health

but this fails with:

Error: unable to health check "temporal.api.workflowservice.v1.WorkflowService" service: connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:7233: connect: connection refused"
('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)

If I SSH into the container, I cannot see 7233 port being bound.

However, if I try the same from outside of the container, I do see it working. I am not even sure how this is working.

[Feature Request] TLS support for postgreSQL

Is your feature request related to a problem? Please describe.

If a PostgreSQL server is enforced SSL connection, temporal-sql-tool should connect with --tls=true.

Currently, the connection from temporal to DB is not a SSL connection. Please find the error below when connecting to a SSL enforced database.

ERROR   Unable to create SQL database.  {"error": "unable to connect to DB, tried default DB names: postgres,defaultdb, errors: [pq: SSL connection is required. Please specify SSL options and retry. pq: SSL connection is required. Please specify SSL options and retry.]", "logging-call-at": "handler.go:100"}

Describe the solution you'd like

Add --tls=${SQL_TLS_ENABLED} for queries in setup_postgres_schema method

Additional context

[Feature Request] Provide ARM64 images for admin-tools and auto-setup

Is your feature request related to a problem? Please describe.

Since the base image(temporalio/base-builder) can already be built with arm64, could you also help provide ARM64 images for admin-tools and auto-setup? We are using it for local testing development, and it's not running well with Macbook M1.

Describe the solution you'd like

Provide arm64 image from https://hub.docker.com/r/temporalio/auto-setup/tags

Additional context

[Feature Request] Prefer release artifacts over building from submodules for the purposes of build container images

Is your feature request related to a problem? Please describe.

This repo depends on submodules to build binaries that are already built and should be artifacts tagged as releases on the respective submodule repos.

Rather than rebuilding this repo should be downloading and extracting the tar ball to the respective container image.

Describe the solution you'd like

Replace submodules with shell script to curl and extract taking in the respective release versions as build args.

Additional context

Temporal repo already emits go-releaser artifacts with release tags.
tcl and ui-server must do the same.
I will take on the work to land those changes in the respective repos and then work to reconcile those changes into this repo.

Create new version of base build to remediate CVE-2022-0318

We have found that the current base-server 1.6.0 has the CVE-2022-0318 for VIM.

Could you please trigger a build for new base images?

[Feature Request] Update admin tools dockerfile to run under non-root user

Please update admin tools dockerfile to run under non-root user like it's done with the server dockerfile.

Create new version of base-builder to remediate CVE-2022-28391

We have found that the current base-builder 1.7.0 has the CVE-2022-28391 for BusyBox.

Could you please trigger a build for new base images?

[Bug] Latest dockerhub images are tagged incorrectly

What are you really trying to do?

Using a latest image of admin-tools, auto-setup or server.

Describe the bug

The latest tag on DockerHub of the above the project incorrectly refers to version v1.7.1 instead of v1.17.1

Minimal Reproduction

Visit:

Additional context

The same had happened to the temporal project, but was fixed, see: temporalio/temporal#3093.

[Bug] Alpine 3.18 includes openssl vulnerabilities

What are you really trying to do?

Our organization deploys Temporal and our internal security scans detected vulnerabilities in temporalio/ui:2.22.0 and earlier.

The origin of them is alpine:3.18. We want to be able to deploy Temporal without these vulnerabilities.

Describe the bug

CVE-2023-5363 detected by Trivy

Minimal Reproduction

trivy image --severity HIGH,CRITICAL temporalio/ui:2.22.0

[Feature Request] Custom Schemas For Auto-Setup (Postgres)

Is your feature request related to a problem? Please describe.

We use Supabase (i.e., built on top of Postgres) as our database provider and backend-as-a-service. As part of their service, they automatically backup (for a price) the postgres database. Currently, we are able to set the database using docker-compose and changing our docker-compose.yml to:

temporal:
    container_name: temporal
    ...
    environment:
      - DB=postgresql
      - DBNAME=postgres
      - VISIBILITY_DBNAME=postgres
      - DB_PORT=5432
      - POSTGRES_USER=postgres
      - POSTGRES_PWD=
      - POSTGRES_SEEDS=db.<supabase-project-id>.supabase.co
      ...

The issue we're facing is that we need to have everything inside of the postgres database, but the auto-setup.sh automatically uses the public schema inside of whatever database is set, without the option to override this via an environment variable set in the docker-compose.yml. We need to be able to set the schema of the temporal database and the temporal visibility database.

Describe the solution you'd like

I would like to be able to pass the environment variables DB_SCHEMA and VISIBILITY_DB_SCHEMA, or something like them (such as just SCHEMA and VISIBILITY_SCHEMA) in the docker-compose.yml file.

These two variables would be passed down to the auto-setup script, and would execute the named schemas when the auto-setup script runs the temporal-sql-tool. It looks like, from here, that the commands would need to be changed to look something like:

SQL_DATABASE=temporal ./temporal-sql-tool --ca search_path=temporal setup-schema -v 0.0

Additional context

It also looks like this change would need to be reflected in the server itself, I just don't know enough about Docker to give nay suggestion. The earlier link seems to suggest it, but I'm just not sure.

Edit 1

From further research, it just looks like there would need to be a way to pass down the --ca flag to the temporal-sql-tool, but I could be wrong. I also saw that the connectAttributes for temporal and visibility will likely need to be changed to reflect the same search_path=SCHEMA_NAME_HERE, but I'm not sure how to incorporate that into this mix.

demz password is logged

see temporalio/temporal#3022

[Bug] Temporal CLI build doesn't set proper version

What are you really trying to do?

Describe the bug

temporal CLI reports version 0.0.0 instead of a proper version

Minimal Reproduction

docker exec temporal-admin-tools temporal -v temporal version 0.0.0 (server 1.20.0) (ui 2.11.0)

Environment/Versions

OS and processor: [e.g. M1 Mac, x86 Windows, Linux]
Temporal Version: [e.g. 1.14.0?] and/or SDK version
Are you using Docker or Kubernetes or building Temporal from source?

Additional context

See temporalio/cli#188 (comment)

[Feature Request] add mysql visibility db independent init params

Is your feature request related to a problem? Please describe.

in my case, databases are managed by a central team, they can't offer db "temporal" and "temporal_visibility" with same dbUrl, userName and password for some reason, it cause problem because the "auto-setup.sh" init db "temporal" and "temporal_visibility" with same env keys. "MYSQL_SEEDS", "MYSQL_USER", "MYSQL_PWD".

Describe the solution you'd like

I notice that when temporal-server generate config file, it has separate 2 db params to different env keys in config_template.yaml with "VISIBILITY_MYSQL_XXX", my solution is to add sub codes to auto-setup.sh

: "${VISIBILITY_MYSQL_SEEDS:=${MYSQL_SEEDS}}"
: "${VISIBILITY_MYSQL_USER:=${MYSQL_USER}}"
: "${VISIBILITY_MYSQL_PWD:=${MYSQL_PWD}}"

...

if [[ ${SKIP_DB_CREATE} != true ]]; then
    temporal-sql-tool --ep "${VISIBILITY_MYSQL_SEEDS}" -u "${VISIBILITY_MYSQL_USER}" -p "${DB_PORT}" "${MYSQL_CONNECT_ATTR[@]}" --db "${VISIBILITY_MYSQL_PWD}" create
fi
temporal-sql-tool --ep "${VISIBILITY_MYSQL_SEEDS}" -u "${VISIBILITY_MYSQL_USER}" -p "${DB_PORT}" "${MYSQL_CONNECT_ATTR[@]}" --db "${VISIBILITY_MYSQL_PWD}" setup-schema -v 0.0
temporal-sql-tool --ep "${VISIBILITY_MYSQL_SEEDS}" -u "${VISIBILITY_MYSQL_USER}" -p "${DB_PORT}" "${MYSQL_CONNECT_ATTR[@]}" --db "${VISIBILITY_MYSQL_PWD}" update-schema -d "${VISIBILITY_SCHEMA_DIR}"

Additional context

I'd like to submit a pr to fix it, may I join the repo

Enable shellcheck as github action

Is your feature request related to a problem? Please describe.

Important Temporal scripts are in this repo now and shellcheck can help to find simple errors.

[Bug] 1.23.1 images don't work on linux/arm64 (Azure)

What are you really trying to do?

I'm trying to run the newest version of temporal server, 1.23.1, on an Azure linux/arm64 virtual machine. There are no issues with running temporal server 1.23.0.

There are no issues with running temporal server 1.23.1 on an Azure linux/amd64 virtual machine.

I have confirmed the issue is present on the following images for 1.23.1

admin-tools
server
auto-setup

Describe the bug

Binaries such as temporal-server, temporal-sql-tool, etc in the 1.23.1 images are failing to start on linux/arm64 machines with the following error: /usr/local/bin/temporal-server: cannot execute binary file: Exec format error

azureuser@temporal:/temporal-test$ docker compose up 
WARN[0000] /temporal-test/docker-compose.yaml: `version` is obsolete 
[+] Running 5/3
 ✔ Network temporal-network        Created                                                                            0.0s 
 ✔ Container temporal-postgresql   Created                                                                            0.1s 
 ✔ Container temporal              Created                                                                            0.1s 
 ✔ Container temporal-ui           Created                                                                            0.1s 
 ✔ Container temporal-admin-tools  Created                                                                            0.1s 
Attaching to temporal, temporal-admin-tools, temporal-postgresql, temporal-ui
temporal-postgresql   | The files belonging to this database system will be owned by user "postgres".
temporal-postgresql   | This user must also own the server process.
temporal-postgresql   | 
temporal-postgresql   | The database cluster will be initialized with locale "en_US.utf8".
temporal-postgresql   | The default database encoding has accordingly been set to "UTF8".
temporal-postgresql   | The default text search configuration will be set to "english".
temporal-postgresql   | 
temporal-postgresql   | Data page checksums are disabled.
temporal-postgresql   | 
temporal-postgresql   | fixing permissions on existing directory /var/lib/postgresql/data ... ok
temporal-postgresql   | creating subdirectories ... ok
temporal-postgresql   | selecting dynamic shared memory implementation ... posix
temporal-postgresql   | selecting default max_connections ... 100
temporal-postgresql   | selecting default shared_buffers ... 128MB
temporal-postgresql   | selecting default time zone ... Etc/UTC
temporal-postgresql   | creating configuration files ... ok
temporal-postgresql   | running bootstrap script ... ok
temporal              | TEMPORAL_ADDRESS is not set, setting it to 172.22.0.3:7233
temporal              | Waiting for PostgreSQL to startup.
temporal-postgresql   | performing post-bootstrap initialization ... ok
temporal-ui           | 2024/05/01 04:55:16 Loading config; env=docker,configDir=config
temporal-ui           | 2024/05/01 04:55:16 Loading config files=[config/docker.yaml]
temporal-ui           | 2024/05/01 04:55:16 Loading config; env=docker,configDir=config
temporal-ui           | 2024/05/01 04:55:16 Loading config files=[config/docker.yaml]
temporal-ui           | 
temporal-ui           |    ____    __
temporal-ui           |   / __/___/ /  ___
temporal-ui           |  / _// __/ _ \/ _ \
temporal-ui           | /___/\__/_//_/\___/ v4.9.0
temporal-ui           | High performance, minimalist Go web framework
temporal-ui           | https://echo.labstack.com
temporal-ui           | ____________________________________O/_______
temporal-ui           |                                     O\
temporal-ui           | ⇨ http server started on [::]:8080
temporal-postgresql   | syncing data to disk ... ok
temporal-postgresql   | 
temporal-postgresql   | 
temporal-postgresql   | Success. You can now start the database server using:
temporal-postgresql   | 
temporal-postgresql   |     pg_ctl -D /var/lib/postgresql/data -l logfile start
temporal-postgresql   | 
temporal-postgresql   | initdb: warning: enabling "trust" authentication for local connections
temporal-postgresql   | You can change this by editing pg_hba.conf or using the option -A, or
temporal-postgresql   | --auth-local and --auth-host, the next time you run initdb.
temporal-postgresql   | waiting for server to start....2024-05-01 04:55:16.500 UTC [48] LOG:  starting PostgreSQL 13.14 (Debian 13.14-1.pgdg120+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
temporal-postgresql   | 2024-05-01 04:55:16.504 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
temporal-postgresql   | 2024-05-01 04:55:16.517 UTC [49] LOG:  database system was shut down at 2024-05-01 04:55:16 UTC
temporal-postgresql   | 2024-05-01 04:55:16.524 UTC [48] LOG:  database system is ready to accept connections
temporal-postgresql   |  done
temporal-postgresql   | server started
temporal-postgresql   | CREATE DATABASE
temporal-postgresql   | 
temporal-postgresql   | 
temporal-postgresql   | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
temporal-postgresql   | 
temporal-postgresql   | 2024-05-01 04:55:16.788 UTC [48] LOG:  received fast shutdown request
temporal-postgresql   | waiting for server to shut down....2024-05-01 04:55:16.793 UTC [48] LOG:  aborting any active transactions
temporal-postgresql   | 2024-05-01 04:55:16.795 UTC [48] LOG:  background worker "logical replication launcher" (PID 55) exited with exit code 1
temporal-postgresql   | 2024-05-01 04:55:16.795 UTC [50] LOG:  shutting down
temporal              | Waiting for PostgreSQL to startup.
temporal-postgresql   | 2024-05-01 04:55:16.833 UTC [48] LOG:  database system is shut down
temporal-postgresql   |  done
temporal-postgresql   | server stopped
temporal-postgresql   | 
temporal-postgresql   | PostgreSQL init process complete; ready for start up.
temporal-postgresql   | 
temporal-postgresql   | 2024-05-01 04:55:16.925 UTC [1] LOG:  starting PostgreSQL 13.14 (Debian 13.14-1.pgdg120+2) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
temporal-postgresql   | 2024-05-01 04:55:16.925 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
temporal-postgresql   | 2024-05-01 04:55:16.925 UTC [1] LOG:  listening on IPv6 address "::", port 5432
temporal-postgresql   | 2024-05-01 04:55:16.932 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
temporal-postgresql   | 2024-05-01 04:55:16.944 UTC [63] LOG:  database system was shut down at 2024-05-01 04:55:16 UTC
temporal-postgresql   | 2024-05-01 04:55:16.952 UTC [1] LOG:  database system is ready to accept connections
temporal              | PostgreSQL started.
temporal              | Setup PostgreSQL schema.
temporal              | /etc/temporal/auto-setup.sh: line 234: /usr/local/bin/temporal-sql-tool: cannot execute binary file: Exec format error
temporal              | /etc/temporal/start-temporal.sh: line 16: /usr/local/bin/temporal-server: cannot execute binary file: Exec format error
temporal              | /etc/temporal/start-temporal.sh: line 16: /usr/local/bin/temporal-server: No error information
temporal exited with code 1

Minimal Reproduction

I am using an Azure linux/arm64 VM, but I believe this is not an Azure specific issue.

On a linux/arm64 machine, run the docker-compose-postgres.yaml on the v.1.23.1 tag of the docker compose samples.

View that the auto-setup container fails to start, with the following logs:

TEMPORAL_ADDRESS is not set, setting it to 172.21.0.3:7233
Waiting for PostgreSQL to startup.
Waiting for PostgreSQL to startup.
PostgreSQL started.
Setup PostgreSQL schema.
/etc/temporal/auto-setup.sh: line 234: /usr/local/bin/temporal-sql-tool: cannot execute binary file: Exec format error
/etc/temporal/start-temporal.sh: line 16: /usr/local/bin/temporal-server: cannot execute binary file: Exec format error
/etc/temporal/start-temporal.sh: line 16: /usr/local/bin/temporal-server: No error information

Environment/Versions

OS and processor: [e.g. M1 Mac, x86 Windows, Linux]

 Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP aarch64 aarch64 aarch64 GNU/Linux linux/arm64

Architecture:           aarch64
  CPU op-mode(s):       32-bit, 64-bit
  Byte Order:           Little Endian
CPU(s):                 4
  On-line CPU(s) list:  0-3
Vendor ID:              ARM
  Model name:           Neoverse-N1
    Model:              1
    Thread(s) per core: 1
    Core(s) per socket: 4
    Socket(s):          1
    Stepping:           r3p1
    BogoMIPS:           50.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
Caches (sum of all):    
  L1d:                  256 KiB (4 instances)
  L1i:                  256 KiB (4 instances)
  L2:                   4 MiB (4 instances)
  L3:                   32 MiB (1 instance)
NUMA:                   
  NUMA node(s):         1
  NUMA node0 CPU(s):    0-3
Vulnerabilities:        
  Gather data sampling: Not affected
  Itlb multihit:        Not affected
  L1tf:                 Not affected
  Mds:                  Not affected
  Meltdown:             Mitigation; PTI
  Mmio stale data:      Not affected
  Retbleed:             Not affected
  Spec rstack overflow: Not affected
  Spec store bypass:    Not affected
  Spectre v1:           Mitigation; __user pointer sanitization
  Spectre v2:           Mitigation; CSV2, BHB
  Srbds:                Not affected
  Tsx async abort:      Not affected

Temporal Version: [e.g. 1.14.0?] 1.23.1
Are you using Docker or Kubernetes or building Temporal from source? Docker

Additional context

Inspecting the docker images of temporalio/server:1.23.0 and temporalio/server:1.23.1, it looks like the build was changed as part of this pull request: #190

temporalio / docker-builds Goto Github PK

docker-builds's People

Contributors

Stargazers

Watchers

Forkers

docker-builds's Issues

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

What are you really trying to do?

Describe the bug

Minimal Reproduction

Additional context

What are you really trying to do?

Describe the bug

Minimal Reproduction

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Edit 1

What are you really trying to do?

Describe the bug

Minimal Reproduction

Environment/Versions

Additional context

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Additional context

Is your feature request related to a problem? Please describe.

What are you really trying to do?

Describe the bug

Minimal Reproduction

Environment/Versions

Additional context

1.23.0

1.23.1

Recommend Projects

Recommend Topics

Recommend Org