sashahart / vex Goto Github PK

I'm considering moving to vex and have lots of virtualenvs that were created with virtualenvwrapper. When activating these virtualenvs, I get:

/home/rsyring/.virtualenvs/hllapi/bin/python: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks. 

If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenv has been installed for
VIRTUALENVWRAPPER_PYTHON=/home/rsyring/.virtualenvs/hllapi/bin/python and that PATH is
set properly.

I guess this is because virtualenvwrapper.py is installed in the global python and vex is switching to the hllapi python before the virtualenvwrapper script is ran. What would be the best way to handle this? Anything vex can do?

I'm getting the following error on Windows (in msysgit's bash and on cmd):

$ vex -mr --python=python myvenv
The executable python (from --python=python) does not exist
Error: error creating virtualenv

$ which python
/c/Python27/python

If I run vex myenv pip install ... it does not install the packages in the virtualenv. It installs them in the main site-packages. A shell starts correctly, though.

Running Windows 7 x64 and Python 2.7.

I love the support for environment variables but I would like to be able specify particular environment variables for particular virtual envs, which doesn't seem to be possible from the config format.

Happy to submit patches if desirable!

I find it really useful to use pipx for managing the user-wide python tools, such as vex.

Installing vex with pipx is slightly more complex than pipx install vex: because vex needs virtualenv as a subprocess, it needs pipx install --include-deps vex, otherwise, the entrypoints of dependencies are not available in the venv vex creates.

Given that pipx is more and more a popular tool among python users, do you think it's worth specifying that in the README?

eval "$(vex --shell-config bash)" prints an error when executed with the OS X (10.9) default bash:

syntax error in conditional expression: unexpected token `('
syntax error near `@($'
`           if [[ "${prev}" != @($take_args) ]]; then'

I think this is because OS X still uses bash 3.2. I don't get this error on Linux with bash 4.3.

I don't have a windows VM to test on, so most of my attempts at windows compatibility are based on guesswork and documentation. Although there is no reason vex should not be able to work in windows, compatibility is likely to be fragile without regular testing.

I don't know of any unencumbered windows VMs to use; EC2 works OK but it's very time consuming to set up a box over RDP and I'm out of pocket to pay for the largish EBS volume windows requires.

Maybe the best solution would be in some third-party testing service, perhaps Travis?

After a fresh install of vex I tried the vex --list but it threw FileNotFoundError: [Errno 2] No such file or directory: '/home/USER/.virtualenvs'

This is correct of course, because there is no such directory but maybe vex should catch this and show an appropriate message.

Vex is an amazing tool, and I'm very interested in using it in a large scale of systems.

But the default ~/.vexrc as a config file location nowadays is really frowned upon. And it's a hardcoded path, with no support for an env var to modify it. Python, Pip, Git and most software use the XDG Base Dirs such as ~/.config and ~/.local for the settings and data.

Would you guys be interested in a patch to also reads from the more standard ~/.config/vex, instead (or in addition to) the current location?

There is an issue with framework builds on Mac OS X. The details are explained perfectly in the following issue in the official virtualenv package: pypa/virtualenv#322

In summary: the environment value __PYVENV_LAUNCHER__ needs to be unset before entering the virtual environment. Here is the commit, that solved the issue in virtualenv: pypa/virtualenv@6793d5f

Saw this on HN, looks really interesting. I've been setting it up but found that the vexrc parsing seems to fail on blank lines which seems a little unforgiving.

I would also recommend switching to yaml as a well defined and readable config format but I'm sure you have your reasons.

Cheers,
Michael

I want to execute blockdiag package in virtualenv called merge in current directory, which does not exist. And vex just seems to execute the command without any virtualenv:

vex --path merge python -m blockdiag

I am on Windows. Is there a way to see what vex is doing? Some kind of -v --verbose flag? I guess I am running vex 0.0.15

Consider a "broken" virtual environment directory which is empty. This actually happended to me - the directory was locked during removal of the environment and was not dropped completely. If vex is asked to install something into such "environment" it can actually install it into the main system.

Replicated like this:

pip uninstall bottle
pip freeze | grep "bottle" # bottle is not installed in the main environment
mkdir ~/.virtualenvs/testenv
vex testenv pip install bottle # should install in testenv or throw error or not install anything...
pip freeze | grep "bottle" # but now we have it in the main environment

I know it's not a very probable situation, but... vex installs a package into the main system, which is not correct. Not sure if this is related to vex, virtualenv or something specific to my environment (checked only on Windows with Git / Bash shell).

For python3, virtualenvs should be created using python -m venv
https://docs.python.org/3/library/venv.html#creating-virtual-environments

But in

I'm trying vez for the first time, I try to run:

vez --shell-config bash

And I get this:

Traceback (most recent call last):
  File "/home/jdevera/.local/bin/vex", line 11, in <module>
    sys.exit(main())
  File "/home/jdevera/.local/lib/python2.7/site-packages/vex/main.py", line 161, in main
    returncode = _main(os.environ, argv)
  File "/home/jdevera/.local/lib/python2.7/site-packages/vex/main.py", line 118, in _main
    return handle_shell_config(options.shell_to_configure, vexrc, environ)
  File "/home/jdevera/.local/lib/python2.7/site-packages/vex/shell_config.py", line 57, in handle_shell_config
    data = shell_config.shell_config_for(shell, vexrc, environ)
  File "/home/jdevera/.local/lib/python2.7/site-packages/vex/shell_config.py", line 48, in shell_config_for
    assert os.path.exists(ve_base)
AssertionError

I only found out that the problem was my missing ~.virtualenvs until I poked in the code. But I'd suggest having a human readable error for this case.

When having an active shell doing: pip install -e git+http://repo/my_project.git#egg=SomeProject installs the package to src in my current working directory. Shouldn't it be installed to the virtual env directory? This is how it works with pyvenv. Maybe my setup is wrong?

This was mentioned to me at PyTexas.

buildout (PyPI: zc.buildout) is something that has existed a long while, which some people have used instead of virtualenv. It is a predecessor and probably in some respects an ancestor of virtualenv. Where virtualenv is just an a la carte, consenting-adults mechanism for isolating python deps to encourage reproducibility, buildout has a broader scope: it's intended a build tool for building whole applications, with a focus on encouraging reproducibility of environments, and fitting somewhere between (e.g.) make and puppet. It uses eggs. It's more about whole workflow than providing just one little piece, as virtualenv does.

I don't have much prior knowledge of buildout's mechanisms. Further, at initial inspection I find the buildout documentation to be relatively challenging. I don't actually even know whether someone hasn't built some buildout extension or workflow which would make vex-like functionality pointless for diehard buildout users.

I suspect given its vintage and emphasis on eggs that buildout is different enough from virtualenv in the concrete specifics that it would be a huge mess to try to include buildout support in vex itself. However, if there is interest I think it could be possible to develop a similar utility to vex, with approximately the same interface, and have that reuse common code from vex-the-package.

I vote for "yes, it should do that".

I was just talking to a friend today, mentioning that I prefer how bundler handles this in the ruby world. You do a bundle install --path .bundle and it then installs everything into that directory. I'd love to have something similar to this.

But I disgress, vex --create and vex --delete or similar things would be amazing. You talked of a spec to someone else on HN, do you want one from me too? (You might already have something in mind, as you already mentioned the whole thing in the README.)

In the past I used Vex with Anaconda and its worked really well. In the last few updates anaconda has changed the the commands instead of just source a file. Conda also changed the way things are set up in the bash profile or bashrc file. Instead of just adding to the path they do this,

 __conda_setup="$('~/anaconda2/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
 eval "$__conda_setup"

When you want to move to different to a different environment you need to do the

conda activate env

I was wondering if there is anyway to change the commands that vex does to set up the virtual environment? I've found a work around with vex but its not as elegant as I hoped.

When I use vex --remove envname, vex starts the virtualenv and then after leaving the environment, it will be removed. Vex should remove the virtual environment without starting it beforehand.

I have the following setuversion p:

• Windows: Windows 10 21H1.
• Python: Installed in C:\Users\sukil\AppData\local\programs\python\python39 (and \python39-32 )
• Vex and virtualenv: Installed user-wide, in C:\users\sukil\appData\roaming\programs\python\python39

When Vex tries to find virtualenv, it uses sys.executable, and appends scripts\virtualenv to its dirname, which results in an error.

Using sys.argv[0] in place of sys.executable and removing the line which says "scripts" solves my problem, but I'm not sure if it's a general enough solution or if anything will break (I think it is, since I cannot run vex indirectly with python -m vex).

I noticed this issue when running vex <venv> ipython.

Normally I can use ctrl-c to cancel the current line of input, and doing this on a blank line just starts a new one, without affecting the interpreter.

However, when running inside vex, the interrupt shuts down IPython instantly, as if I'd pressed ctrl-d. My guess is that the parent process is not handling the SIGINT signal, and its shutdown results in the child (ipython) shutting down as well, even though it normally handles interrupts gracefully.

Can this be fixed? It really makes interactive sessions in IPython/vex unusable for me, since I have a habit of cancelling half-constructed lines of code by interrupting them.

Thanks!

How come vex is using Popen to run the subprocess (instead of os.execvp or something) on *nix? (not on Windows obviously 😁)

This will invariably create problems with signal handling and getting a handle to the actual pid ...

I'm interested in hearing why subprocess.Popen was used over os.execvp.

I'd like to see a -t option that would essentially do this:

vex -mr tmp-$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 4 | head -n 1)

Example usage:

$ vex -t
(tmp-s3a8) $ 

$ ~/.local/bin/vex test zsh -f -d
$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/<me>/.virtualenvs/test/bin

I renamed all of my .z* files, and ran zsh -f -d, which ensures that zsh doesn't load RC files. In others words, this starts zsh as cleanly as I can, ignoring all of my personal and OS config. And yet, my virtualenv's bin still ends up at the end of my PATH instead of at the beginning.

berdario/pew supports setproject which is a nice convenience to cd into a project folder associated with a venv activation.

I like vex's shorter command to activate venv (instead od pew workon) but it meas I have to cd use a second command to enter project folder.

It would be nice if vex checked for .project file existence and entered the project folder upon subshell activation.

So not to interrupt the workflow that people may be relying upon, but I feel like the following is typical for my workflow...

$ vex -m --python python<Version> <virtualenv name>
(vex:<venv>)$ # do stuff
(vex:<venv>)$ exit
$ vex -r <virtualenv name>
(vex:<venv>)$ exit

It's minor, but I'd like to have the ability to just run, without entering the virtual environment:

$ vex -r <virtualenv name>
<venv destroyed>
$

And I think it might be worthwhile to not necessarily step into the environment every time:

$ vex -m --python python<Version> <virtualenv name>
$ vex <virtual name> <command-line>
$ vex -r <virtualenv name>

So the proposal is to add a new command-line argument: -X or --exit

Functionally, this may mean that the virtual environment is never entered (i.e don't run Popen).

Using my example above:

$ vex -mX --python python<Version> <virtualenv name>
$ vex <virtual name> <command-line>
$ vex -rX <virtualenv name>

When I use "shell=bash" in my .vexrc file, and type the following commands:
$ vex myenv
$ pip freeze

everything is ok, but if I use "shell=zsh", and type the same commands, I get the packages list installed in my system, the same list if I do not use the vex/virtualenvs.

Mac OS X 10.10
Tried with fish and bash
Why would this be happening?

$ vex --make test
Traceback (most recent call last):
  File "/usr/local/bin/vex", line 9, in <module>
    load_entry_point('vex==0.0.15', 'console_scripts', 'vex')()
  File "/Library/Python/2.7/site-packages/vex/main.py", line 161, in main
    returncode = _main(os.environ, argv)
  File "/Library/Python/2.7/site-packages/vex/main.py", line 133, in _main
    handle_make(environ, options, make_path)
  File "/Library/Python/2.7/site-packages/vex/make.py", line 28, in handle_make
    os.mkdir(ve_base)
OSError: [Errno 2] No such file or directory: '/Users/user/~/.virtualenv'

The tilde is not being substituted, and is being added to pwd

e.g. vex.run.BadConfigError: ve_bin '/tmp/blank/bin' does not exist
when pointed at /tmp/blank

If a virtual environment is broken (for example, Ctrl-C is pressed at the beginning of its creation) and not all the folders typical for environment are there (example: we have only Lib folder inside ~/.virtualenvs/env) it is not possible to remove such environment with vex.

vex -r env mycommand
Error: path 'C:\Users\bartgo.virtualenvs\env' did not look like a virtualenv

I am using vex in a bash script and I would like it to drop the environment in any case to guarantee that the environment is empty before installing there things with pip.

Would you please consider implementing a force (-f / --force) flag to be used like vex -rf env mycommand or vex -rmf env command or vex --remove --force env mycommand to force removal of a virtual environment, even if it's broken?

On Python 3.4 when trying to run vex on Windows with python -m vex I get the error No module named vex.__main__; 'vex' is a package and cannot be directly executed.

This is because I'm running vex as a script which has no __main__ method. I can't run just vex from the command line because it's not on the path. I think it's something to do with the new venv in Python 3.4.

I know you have advice to add the vex script to the PATH environment, but I think most people just want things to be fully installed and ready to go when they do pip install vex. Do you have any good idea of how to go about that?

I read the other issues and it looks like you're looking for some help with supporting Windows. I'd be happy to help - it sounds like some of the Windows-related stuff could deal with some simplification.

I now get this error when I use vex:

/usr/local/bin/vex:5: UserWarning: Module _markerlib was already imported from /Library/Python/2.7/site-packages/distribute-0.6.49-py2.7.egg/_markerlib/__init__.pyc, but /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python is being added to sys.path
  from pkg_resources import load_entry_point

Any ideas?

I frequently run into the situation where I run "command1", realize I'm not in a virtualenv, run vex somevenv to get a new bash working in the venv, and then hit the up arrow key to get to "command1" in bash history, only to realize that the original bash process has not exited so "command1" isn't in the new bash process.

Maybe this is just a bash thing and not in the pervue of vex. If so, feel free to close. :)

If you use homebrew on OS X, you have to add something like export PATH=/usr/local/bin:$PATH to your .bashrc to allow, e.g., a homebrewed Python to shadow the system Python.

When I run vex venv bash, the path to the venv is prepended to $PATH before the .bashrc is read, so I end up with a $PATH like this: /usr/local/bin:~/.virtualenvs/venv/bin:$PATH. But it should be: ~/.virtualenvs/venv/bin:/usr/local/bin:$PATH.

I think this is no problem in virtualenvwrapper, because I source the virtualenvwrapper.sh in my .bashrc after I prepend /usr/local/bin for homebrew to my $PATH.

On Windows 7, when I run the python interpreter with no arguments with Vex, I get no history (nothing happens when I arrow up). I've tried running cmd /c python and cmd /k python with no difference.
I assume this is a cabeat, but I couldn't find any explanations as to why it happens in the readme.

>vex foo python -c "import sys; print(sys.executable)"
C:\Apps\Python34\python.exe

I suspect this is because the wrapper vex.exe runs python.exe to run the vex code, and then that runs the command python with an updated PATH. But Windows always looks in the directory of the executable before searching PATH, and so find the system python.exe rather than the one in the virtualenv (see http://msdn.microsoft.com/en-gb/library/windows/desktop/ms682425(v=vs.85).aspx for details).

I don't know of an easy fix for this, other than manually parsing the command line and doing the PATH search explicitly, instead of letting Windows do it.

The same problem will occur with executables stored alongside the system Python: vex foo pip list lists what's installed in the system Python, not what's in the virtualenv :-(

I suppose this is probably a feature request, but I'd like to be able to do something like the following:

export SPARK_HOME=/usr/lib/spark
export SPARK_PYTHONPATH=$SPARK_HOME/python
export PYTHONPATH=$SPARK_PYTHONPATH:$PYTHONPATH
export PYTHONPATH=$(find $SPARK_PYTHONPATH/lib/py4j*.zip):$PYTHONPATH

Right now, the env section is a direct text replacement, which makes something like the above fairly impossible. I end up polluting my standard environment so I can pull it into my vex environment. That's probably okay for the example above on a system I control completely. It's probably undesirable from a system standpoint where I'd like to keep my venv and other separate.

From virtualenvwrapper docs as one of it's features:

1. Organizes all of your virtual environments in one place.

I've seen some pythonistas, who use barebones virtualenv, preach the benefits of storing each virtualenv in the corresponding project's root. It makes a lot of sense to me:

1. You don't have to name your env when you use vex, don't need to remember, and it's faster to use.
2. You don't end up with orphan envs in ~/.virtualenvs after deleting the project they belong to.
3. All files related with a project, are contained inside that project's directory.

Once decentralised mode is activated in ~/.vexrc, you wouldn't need to specify which env you want to use anymore. Vex looks in the directory you are working on, kinda like git does. Examples:

• vex --make creates a ./virtualenv in the current directory.
• vex --remove removes the ./virtualenv in the current directory.
• vex (iPython) spawns a shell (or iPython) using the ./virtualenv in the current directory. If one is not found, vex searches the parent directory recursively until it finds one.
• vex --env <nameOfEnv> would allow you to quickly tap the traditional centralised directory, by specifying the name of the env using the --env flag.

Does something like this make any sense to you?

I've started looking at vex as a possible replacement for the virtualenvwrapper shell commands. I really like it (and am suggesting we add a reference from packaging.python.org: pypa/packaging.python.org#118)

However, the one command I particularly miss is "lsvirtualenv" - it would be handy if vex included a "--list" option to show the currently available named environments.