Code Monkey home page Code Monkey logo

infrabox-cli's Introduction

REUSE status

InfraBox CLI

With the InfraBox CLI you can run your InfraBox jobs on your local machine and configure your project.

Install

To install infraboxcli you need to have these requirements already installed:

  • git
  • docker
  • python & pip

Then simply run:

pip install infraboxcli

You can validate your installation by running:

infrabox version

List Jobs

If you have a more complex project it may be helpful to list all available jobs in it. For this you may use:

infrabox list

It outputs the names of all available jobs. An example output may look like this:

tutorial-1
tutorial-1/step1
tutorial-1/step2
tutorial-1/step3
tutorial-1/step4
tutorial-1/step5
tutorial-1/tutorial-1/step1/tests
tutorial-1/tutorial-1/step2/tests
tutorial-1/tutorial-1/step3/tests
tutorial-1/tutorial-1/step4/tests
tutorial-1/tutorial-1/step5/tests

Run a Job

InfraBox CLI may be used to run you jobs on your local machine. It will also respect all the dependencies and run the jobs in the correct order. Available options are:

usage: infrabox run [-h] [--no-rm] [-t TAG] [--local-cache LOCAL_CACHE]
                    [job_name]

positional arguments:
  job_name              Job name to execute

optional arguments:
  -h, --help            show this help message and exit
  --no-rm               Does not run 'docker-compose rm' before building
  -t TAG                Docker image tag
  --local-cache LOCAL_CACHE
                        Path to the local cache

To run all jobs defined in your infrabox.json file simply do:

infrabox run

In case you have multiple jobs defined an want to run only one of them you can do:

infrabox run <job-name>

Push a Job

To be able to use infrabox push you have to create a project in the InfraBox Dashboard and create an auth token for it.

Auth Token and InfraBox API Host must be set as environment variables.

export INFRABOX_CLI_TOKEN=<YOUR_ACCESS_TOKEN>
export INFRABOX_URL=<INFRABOX_URL>

To push your local project simply do:

infrabox push

This will compress your local project and upload it to InfraBox. Now you can open the InfraBox Dashboard and navigate to your project. You should see the jobs running on InfraBox.

You can also watch the console output of your pushed jobs locally. Just use the --show-console option.

infrabox push --show-console

Pull a Job

In case you would like to run a job which has been already executed on InfraBox you can use infrabox pull. It will download the docker container and all its inputs so you can the same container locally and investigate any issue.

infrabox pull --job-id <JOB_ID>

You can find the exact command for each job on the job detail page of InfraBox under Run local

Secrets

If you reference secrets in your job definition (i.e. as environment variable) then you can add a .infraboxsecrets.json file to your project right next to the .infrabox.json file. This file should then contain all your secrets referenced in your job definition as a simple object:

{
    "SECRET_NAME1": "my secret value",
    "Another secret": "another value"
}

How to get support

If you need help please post your questions to Stack Overflow. In case you found a bug please open a Github Issue. Follow us on Twitter: @Infra_Box or have look at our Slack channel infrabox.slack.com.

infrabox-cli's People

Contributors

chengshifan avatar d-johannsen avatar epdhowwd avatar erikbsap avatar fspaniol avatar hendrikr avatar ib-franziska avatar ib-steffen avatar iberryful avatar jeffcfbr avatar linuo0 avatar meranos avatar metti avatar sap-steffen avatar sebastianwolf-sap avatar sept-en avatar steffen911 avatar wamdo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

infrabox-cli's Issues

infraboxcli -f param is not working

When use -f to specify other infrabox.json file like

infrabox -f infrabox_debug.json push

It still use the default infrabox.json as the workflow.

Here's my infraboxcli version:

infraboxcli 0.6.6

Invalid property 'enable_docker_build_kit'

I'm trying to run the command infrabox list using infrabox-cli 0.9.0, but I keep getting the error:
% infrabox list
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/bin/infrabox", line 3, in
infraboxcli.main()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/infraboxcli/init.py", line 296, in main
args.func(args)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/infraboxcli/list_jobs.py", line 18, in list_jobs
data = load_infrabox_file(args.infrabox_file_path)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/infraboxcli/job_list.py", line 30, in load_infrabox_file
validate_json(data)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyinfrabox/infrabox/init.py", line 524, in validate_json
parse_document(d)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyinfrabox/infrabox/init.py", line 521, in parse_document
parse_jobs(d['jobs'], "#jobs")
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyinfrabox/infrabox/init.py", line 506, in parse_jobs
parse_docker(elem, p)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyinfrabox/infrabox/init.py", line 272, in parse_docker
check_allowed_properties(d, path, ("type", "name", "docker_file", "depends_on", "resources",
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pyinfrabox/utils.py", line 33, in check_allowed_properties
raise ValidationError('%s(%s)' % (path, d['name']), "invalid property '%s'" % key)
pyinfrabox.ValidationError: #jobs0: invalid property 'enable_docker_build_kit'

Context not set when using image in docker-compose

When having a job to build an image and referencing it on another, the context from the image is not imported. This differs from Infrabox itself.

A proposed workaround is to COPY the needed files in the Dockerfile.

Client does not work on mac

When trying to run any command on MAC, I get the following:

 infrabox version
Traceback (most recent call last):
  File "/usr/local/bin/infrabox", line 2, in <module>
    import infraboxcli
  File "/usr/local/lib/python2.7/site-packages/infraboxcli/__init__.py", line 13, in <module>
    from infraboxcli.install import install_infrabox
  File "/usr/local/lib/python2.7/site-packages/infraboxcli/install.py", line 11, in <module>
    from halo import Halo
  File "/usr/local/lib/python2.7/site-packages/halo/__init__.py", line 7, in <module>
    from .halo import Halo
  File "/usr/local/lib/python2.7/site-packages/halo/halo.py", line 17, in <module>
    from halo._utils import is_supported, colored_frame, is_text_type, decode_utf_8_text, get_terminal_columns, \
  File "/usr/local/lib/python2.7/site-packages/halo/_utils.py", line 7, in <module>
    from backports.shutil_get_terminal_size import get_terminal_size
ImportError: No module named shutil_get_terminal_size

Windows: [infrabox] Job failed: environment can only contain strings

This happens with docker-compose and may also happen with docker (non-compose) on the Windows operating system.

I used a workaround in run.py to encode the secret and non-secret names and values to ascii rather than unicode. I am not sure if this is the correct fix or not.

e.g.
env[name.encode('ascii', 'ignore')] = get_secret(args, value['$secret']).encode('ascii', 'ignore')

Error:
Traceback (most recent call last):
File "c:\Python27\lib\site-packages\infraboxcli\run.py", line 405, in build_and_run
build_and_run_docker_compose(args, job)
File "c:\Python27\lib\site-packages\infraboxcli\run.py", line 234, in build_and_run_docker_compose
ignore_output=False)
File "c:\Python27\lib\site-packages\infraboxcli\execute.py", line 18, in execute
universal_newlines=True)
File "c:\Python27\lib\subprocess.py", line 394, in init
errread, errwrite)
File "c:\Python27\lib\subprocess.py", line 644, in _execute_child
startupinfo)
TypeError: environment can only contain strings
[infrabox] Job failed: environment can only contain strings

jobs of type `docker` with `deployment` should already tag the image locally

When defining jobs of type docker with deployment it would be handy if runing the job already tags the image for deployment. It is clear that a local execution should not push the image to the target registry. However, if the image is at least tagged locally, a follow-up job would be able to use the image under the deployment name already.

Example:

    {
        "name": "build-installer-test-base",
        "type": "docker",
        "docker_file": "infrabox/test-install/Dockerfile",
        "build_context": "../..",
        "build_only": true,
        "resources": { "limits": { "cpu": 1, "memory": 2048 } },
        "deployments": [{
            "type": "docker-registry",
            "host": "v2-registry.dhcp.wdf.sap.corp",
            "repository": "infrabox/datahub-operator/com.sap.datahub.linuxx86_64/installer-test-base"
        }]
    },{
        "name": "test-install",
        "type": "docker-image",
        "image": "v2-registry.dhcp.wdf.sap.corp/infrabox/datahub-operator/com.sap.datahub.linuxx86_64/installer-test-base:build_${INFRABOX_BUILD_NUMBER}",
        "resources": { "limits": { "cpu": 1, "memory": 1024 } },
        "depends_on": ["build-installer-test-base"]
    }

infrabox run build-installer-test-base creates datahub-operator_build-installer-test-base:latest, but not v2-registry.dhcp.wdf.sap.corp/infrabox/datahub-operator/com.sap.datahub.linuxx86_64/installer-test-base:build_local`.

infrabox run test-install cannot find its image, as the image was not tagged.

--build-args not passed to docker-compose

I see the --build-arg arguments transferred to the docker build command infrabox is running. But I don't see it transferred to the docker-compose command infrabox runs for a docker-compose job.

In both cases I am running infrabox run --build-arg HTTP_PROXY="$HTTP_PROXY" VFLOW_VERSION=2.3.55 SUBCPP_VERSION=2.1.0

Generated docker build statement = docker build -t datahub-dq-integration_smoke-tests . -f /SAPDevelop/go/src/github.wdf.sap.corp/bdh/datahub-dq-integration/eim-vflow-operators/tests/vflow_server/Dockerfile --build-arg HTTP_PROXY=http://proxy.lse.sap.corp:8080 --build-arg VFLOW_VERSION=2.3.55 --build-arg SUBCPP_VERSION=2.1.0 -m 2048m

Generated docker-compose statement = docker-compose -p datahub-dq-integration -f /SAPDevelop/go/src/github.wdf.sap.corp/bdh/datahub-dq-integration/eim-vflow-operators/tests/docker-compose.yml.infrabox build

Is there another way to get the proxy information to docker-compose

Infraboxcli login not working on Python 3

Seems like a raw_input call is used, which only works in Python 2 (should be input in Python 3).

Traceback (most recent call last):
  File "/Users/jeremy/.pyenv/versions/arch-3.6/bin/infrabox", line 3, in <module>
    infraboxcli.main()
  File "/Users/jeremy/.pyenv/versions/3.6.7/envs/arch-3.6/lib/python3.6/site-packages/infraboxcli/__init__.py", line 296, in main
    args.func(args)
  File "/Users/jeremy/.pyenv/versions/3.6.7/envs/arch-3.6/lib/python3.6/site-packages/infraboxcli/dashboard/user.py", line 32, in login
    email = raw_input("Email: ")
NameError: name 'raw_input' is not defined

Job of type `docker` ignore `command` option

It seems infraboxcli 0.8.4. ignores the command option for jobs of type docker.
Running the following job with infrabox run does not show the text given in the command part:

  • infrabox.json:
{
    "version": 1,
    "jobs": [
        {
            "name": "boxtest-docker",
            "type": "docker",
            "build_only": false,
            "docker_file": "Dockerfile",
            "command" : ["echo", "I am of type docker (plain)"],
            "resources": { "limits": { "memory": 1024, "cpu": 1 } }
        }
    ]
}
  • Dockerfile
FROM library/alpine

"No module named '_curses'" after fresh install

I just did

$ pip install infraboxcli
$ infrabox version

and got

An error was found, but returning just with the version: No module named '_curses'
infraboxcli 0.8.7

(using infrabox on a Windows PC)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.