Comments (8)
Looks like we're missing https://goreleaser.com/cookbooks/multi-platform-docker-images/ in our CLI Go releaser Docker configuration
cloudquery/cli/.goreleaser.yaml
Line 13 in 5d8cd11
from cloudquery.
Hi @JumaX, thanks for the issue. We've just released a multi-arch image of the CLI under ghcr.io/cloudquery/cloudquery:5.1.1
(also ghcr.io/cloudquery/cloudquery:5.1
).
That should work for both amd64
and arm64
. You can also try the arm64
image directly via ghcr.io/cloudquery/cloudquery:5.1.1-linux-arm64
.
Please let us know if that works.
from cloudquery.
@erezrokah Thanks for this. Image now runs, however it exits immediately, as I believe their isn't a process running to keep it up. I tried a quick command: tail -f /dev/null
but tail isn't present in the image. Are you able to update the image to have it stay up by default.
from cloudquery.
How are you running the image?
Can you share the logs from the container?
This is what I see and expected without any arguments:
docker run ghcr.io/cloudquery/cloudquery:5.1.1
CloudQuery CLI
Open source data integration at scale.
Find more information at:
https://www.cloudquery.io
Usage:
cloudquery [command]
Available Commands:
addon Addon commands
login Login to CloudQuery Hub.
logout Log out of CloudQuery Hub.
migrate Update schema of your destinations based on the latest changes in sources from your configuration
plugin Plugin commands
switch Switches between teams.
sync Sync resources from configured source plugins to destinations
tables Generate documentation for all supported tables of source plugins specified in the spec(s)
Flags:
--cq-dir string directory to store cloudquery files, such as downloaded plugins (default ".cq")
-h, --help help for cloudquery
--log-console enable console logging
--log-file-name string Log filename (default "cloudquery.log")
--log-format string Logging format (json, text) (default "text")
--log-level string Logging level (default "info")
--no-log-file Disable logging to file
--telemetry-level string Telemetry level (none, errors, stats, all) (default "all")
-v, --version version for cloudquery
Use "cloudquery [command] --help" for more information about a command.
from cloudquery.
I am using docker compose to spin up a local environment, to go postgres to postgres to better understand the product.
For a container to keep running their needs to be a long running process. It looks like there isn't one in this container, thus it exits, which is expected behaviour for docker.
Here is a sample config and output:
start containers with: docker-compose up -d
✔ Network cloud-query-interview_default Created 0.0s
✔ Container postgres_db_source Started 0.0s
✔ Container postgres_db_destination Started 0.0s
✔ Container cloudquery Started
Then see that the cloudquery container exits.
Sample docker-compose file:
version: "3.8"
services:
cloudquery_demo:
image: ghcr.io/cloudquery/cloudquery:latest-linux-arm64
platform: linux/arm64
container_name: cloudquery
volumes:
- redacted
- redacted
ports:
- "7777:7777"
depends_on:
- db_source
- db_destination
db_source:
image: postgres:15.5
container_name: postgres_db_source
environment:
POSTGRES_USER: redacted
POSTGRES_PASSWORD: redacted
POSTGRES_DB: redacted
volumes:
- ./postgres_data_source:/var/lib/postgresql/data/
- ./init-db-source.sh:/docker-entrypoint-initdb.d/init-db.sh
ports:
- "5432:5432"
db_destination:
image: postgres:15.5
container_name: postgres_db_destination
environment:
POSTGRES_USER: redacted
POSTGRES_PASSWORD: redacted
POSTGRES_DB: redacted
volumes:
- ./postgres_data_destination:/var/lib/postgresql/data/
- ./init-db-destination.sh:/docker-entrypoint-initdb.d/init-db.sh
ports:
- "5433:5432"
from cloudquery.
Can you share the (redacted) container logs? It's hard knowing what the issue without seeing the error reported by the CLI.
Does docker run ghcr.io/cloudquery/cloudquery:5.1.1
give you the CLI help?
from cloudquery.
There is no error to be clear, the container itself is not built correctly, unless you are saying that you expect the container to run the CLI once and then exit?
In which case, does that mean I have to re-run the container every time I want to move some data around or change the configuration?
I would generally expect for there to be a long running service to interact with in the container that I can interact with (CLI/API etc) to update configurations, change data sources etc.
from cloudquery.
Hi @JumaX, the CLI usually runs the sync once then exit, unless you use CDC, which I think that's what you're trying to do correct?
If that's the case it should be a long running process.
There is no error to be clear, the container itself is not built correctly
Is there a build error, what do you mean by not built correctly?
would generally expect for there to be a long running service to interact with in the container that I can interact with (CLI/API etc) to update configurations, change data sources etc.
I'm not sure what you mean by that. Maybe it would make things simpler to try the quickstart guide first to run a "simple" sync. The CLI doesn't expose an API to change the configuration mid-sync, you should be able to restart the container with a new spec to do that
Regardless, I think this discussion it out of scope of the original issue.
Do you mind opening a new one? Please share the information as requested in the issue template, specifically the configuration content that's passed to the CLI
from cloudquery.
Related Issues (20)
- feat: Support Azure storage account access keys resource
- Validate Snowflake DSN
- bug: aws_s3_buckets does not specify Server Access log format HOT 3
- bug: Github Plugin not returning repo custom properties HOT 3
- bug: Jira, not showing issue parentage HOT 4
- feat: PostgreSQL destination delete stale requires sequential scan HOT 3
- bug: Postgres source is not recognizing partitioned tables
- feat: Log failed message on column resolver exception in Plugin SDK
- bug: Pagerduty schedules table contains blank columns for schedule layers
- Bug: unix domain socket on Windows HOT 4
- feat: expose scaleway vpcs HOT 1
- Unable to use incremental sync on Windows HOT 1
- feat: Add list secrets for vault HOT 3
- bug: ORACLE plugin: no IP address info in oracle_virtualnetwork_ephemeral_public_ips and oracle_virtualnetwork_reserved_public_ips resource table HOT 3
- bug: ORACLE plugin: no data in oracle_virtualnetwork_private_ips resource table HOT 5
- bug: `--log-console` is broken on CLI version `v5.20.4` HOT 1
- bug: bitbucket plugin misleading installation docs HOT 1
- bug: Referencing a single relation table might include other non related relation tables
- bug: Unclear error message(s) when using the CLI without selecting a team
- bug(tests): Occasional data race reported in the tests
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cloudquery.