Pass Parameters at Runtime? about bonobo HOT 9 CLOSED

Anybody?

from bonobo.

For now, the recommended way to pass arguments at runtime is to use environment variables. It's universal, widely used, and ops-friendly.

There will be in the future some way to use arguments to set it, but I see it as a non priority knowing that env can do pretty much anything in this regard.

Do you have a particuliar use case where environment cannot do the job ?

INPUT_FILE=inventory.txt MAGIC_NUMBER=2773 bonobo run csvsanitizer

from bonobo.

Thanks for getting back to me!

Believe it or not I didn't know you could do that with bash. Unfortunately, my work environment is exclusively windows excepting our FTP and proxy servers themselves. I would need to be using one of our Windows Server 2012 production servers so I can't even get around it via the Linux subsystem for Windows (Ubuntu Bash on Windows).

Although I might just not be aware of how, I am not sure you can set env vars before the bonobo run command like you can in bash. I have seen suggestions for ways to get-around this issue on stack exchange though it looks to be pretty hacky and not ideal (https://superuser.com/questions/223104/setting-and-using-variable-within-same-command-line-in-windows-cmd-exe). Additionally, even setting them in separate steps wouldn't work well as, in my experience at least, unlike Linux environment variables they don't seem to be as easily set or changed, especially for individual shell instances.

I realize that docker is an option but it still seems like extra complexity for most of my use cases where I really just want to pass traditional kwargs. Additionally, Am I right in believing, especially after your answer, that you not implementing kwargs was a design decision, or do you just believe it unnecessary? It is your project so I completely understand if it is the former but I would love to do some work on it and submit a pull request if it is the latter and you are open to the idea.

Lastly, perhaps you know something about Windows which I don't and your suggestion would work, albeit with minor variations. Do you?

Once again, thanks for getting back to me!

from bonobo.

Ok, I'm not very familiar with windows (understand, I did not use a windows machine more than a couple times for the last 10 years), but I guess you have a very valid point, I tend to think bash is universal, but indeed it's not.

I'll think about an alternate solution, which will probably be an additional command line flag that sets an environment variable (for exemple, -e FOO=bar), so windows or any strange env will work. Also, support for a ".env" file will be present in the near future.

I created a draft "rfc" about that to summarize all said here and to draft the future features and documentation on this topic: https://github.com/python-bonobo/bonobo/wiki/RFC-Configuration-&-Environment

from bonobo.

Thanks Romain, I think that sounds great.

from bonobo.

If you want to submit a pull request in this direction like you suggested (using argparse, and using same syntax as docker does: https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e-env-env-file), I'll be very glad to integrate it in the next release.

from bonobo.

Romain,

Absolutely, I would be happy to work on that.

from bonobo.

Checklist:

• Open PR against python-bonobo/bonobo:develop
• Move tests/util/get_passed_env.py into examples
• Documentation page about passing env. That can be a new docs/guide/environment.rst (or something else if you think of a better name) file, that will be linked from the main guide index.
• --env argparse registration may want to go into register_generic_run_arguments instead of register, as environment will probably be usefull in more than just bonobo run context

from bonobo.

@hartym,

Am I right in believing now that your merged my pull request into the develop branch you want me to pull down a new copy of the develop branch, go through the checklist you provided, and then submit a new pull request against the develop branch?

Additionally, I am fine with a new 'environment' doc page but when are you planning on fleshing-out http://docs.bonobo-project.org/en/master/reference/commands.html? I would be happy to do one or both depending on which you think is better. However, regarding the latter I would like to know what the -c cmd is for, are you still planning on implementing it, if so, could you provided an example?

Thanks

from bonobo.