Code Monkey home page Code Monkey logo

Comments (8)

erezrokah avatar erezrokah commented on June 14, 2024

Looks like we're missing https://goreleaser.com/cookbooks/multi-platform-docker-images/ in our CLI Go releaser Docker configuration

from cloudquery.

erezrokah avatar erezrokah commented on June 14, 2024

Hi @JumaX, thanks for the issue. We've just released a multi-arch image of the CLI under ghcr.io/cloudquery/cloudquery:5.1.1 (also ghcr.io/cloudquery/cloudquery:5.1).

That should work for both amd64 and arm64. You can also try the arm64 image directly via ghcr.io/cloudquery/cloudquery:5.1.1-linux-arm64.

Please let us know if that works.

from cloudquery.

JumaX avatar JumaX commented on June 14, 2024

@erezrokah Thanks for this. Image now runs, however it exits immediately, as I believe their isn't a process running to keep it up. I tried a quick command: tail -f /dev/null but tail isn't present in the image. Are you able to update the image to have it stay up by default.

from cloudquery.

erezrokah avatar erezrokah commented on June 14, 2024

How are you running the image?
Can you share the logs from the container?

This is what I see and expected without any arguments:

docker run ghcr.io/cloudquery/cloudquery:5.1.1
CloudQuery CLI

Open source data integration at scale.

Find more information at:
	https://www.cloudquery.io

Usage:
  cloudquery [command]

Available Commands:
  addon       Addon commands
  login       Login to CloudQuery Hub.
  logout      Log out of CloudQuery Hub.
  migrate     Update schema of your destinations based on the latest changes in sources from your configuration
  plugin      Plugin commands
  switch      Switches between teams.
  sync        Sync resources from configured source plugins to destinations
  tables      Generate documentation for all supported tables of source plugins specified in the spec(s)

Flags:
      --cq-dir string            directory to store cloudquery files, such as downloaded plugins (default ".cq")
  -h, --help                     help for cloudquery
      --log-console              enable console logging
      --log-file-name string     Log filename (default "cloudquery.log")
      --log-format string        Logging format (json, text) (default "text")
      --log-level string         Logging level (default "info")
      --no-log-file              Disable logging to file
      --telemetry-level string   Telemetry level (none, errors, stats, all) (default "all")
  -v, --version                  version for cloudquery

Use "cloudquery [command] --help" for more information about a command.

from cloudquery.

JumaX avatar JumaX commented on June 14, 2024

I am using docker compose to spin up a local environment, to go postgres to postgres to better understand the product.

For a container to keep running their needs to be a long running process. It looks like there isn't one in this container, thus it exits, which is expected behaviour for docker.

Here is a sample config and output:

start containers with: docker-compose up -d

 ✔ Network cloud-query-interview_default  Created                                                                                                                                   0.0s 
 ✔ Container postgres_db_source           Started                                                                                                                                   0.0s 
 ✔ Container postgres_db_destination      Started                                                                                                                                   0.0s 
 ✔ Container cloudquery                   Started   

Then see that the cloudquery container exits.
Screenshot 2024-01-24 at 19 08 56 Screenshot 2024-01-24 at 19 09 22

Sample docker-compose file:

version: "3.8"
services:
  cloudquery_demo:
    image: ghcr.io/cloudquery/cloudquery:latest-linux-arm64
    platform: linux/arm64
    container_name: cloudquery
    volumes:
      - redacted
      - redacted
    ports:
      - "7777:7777"
    depends_on:
      - db_source
      - db_destination

  db_source:
    image: postgres:15.5
    container_name: postgres_db_source
    environment:
      POSTGRES_USER: redacted
      POSTGRES_PASSWORD: redacted
      POSTGRES_DB: redacted
    volumes:
      - ./postgres_data_source:/var/lib/postgresql/data/
      - ./init-db-source.sh:/docker-entrypoint-initdb.d/init-db.sh
    ports:
      - "5432:5432"

  db_destination:
    image: postgres:15.5
    container_name: postgres_db_destination
    environment:
      POSTGRES_USER: redacted
      POSTGRES_PASSWORD: redacted
      POSTGRES_DB: redacted
    volumes:
      - ./postgres_data_destination:/var/lib/postgresql/data/
      - ./init-db-destination.sh:/docker-entrypoint-initdb.d/init-db.sh
    ports:
      - "5433:5432"

from cloudquery.

erezrokah avatar erezrokah commented on June 14, 2024

Can you share the (redacted) container logs? It's hard knowing what the issue without seeing the error reported by the CLI.

Does docker run ghcr.io/cloudquery/cloudquery:5.1.1 give you the CLI help?

from cloudquery.

JumaX avatar JumaX commented on June 14, 2024

There is no error to be clear, the container itself is not built correctly, unless you are saying that you expect the container to run the CLI once and then exit?

In which case, does that mean I have to re-run the container every time I want to move some data around or change the configuration?

I would generally expect for there to be a long running service to interact with in the container that I can interact with (CLI/API etc) to update configurations, change data sources etc.

from cloudquery.

erezrokah avatar erezrokah commented on June 14, 2024

Hi @JumaX, the CLI usually runs the sync once then exit, unless you use CDC, which I think that's what you're trying to do correct?

If that's the case it should be a long running process.

There is no error to be clear, the container itself is not built correctly

Is there a build error, what do you mean by not built correctly?

would generally expect for there to be a long running service to interact with in the container that I can interact with (CLI/API etc) to update configurations, change data sources etc.

I'm not sure what you mean by that. Maybe it would make things simpler to try the quickstart guide first to run a "simple" sync. The CLI doesn't expose an API to change the configuration mid-sync, you should be able to restart the container with a new spec to do that

Regardless, I think this discussion it out of scope of the original issue.

Do you mind opening a new one? Please share the information as requested in the issue template, specifically the configuration content that's passed to the CLI

from cloudquery.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.