sayanarijit / expandvars Goto Github PK

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

The bash manual has a list of some more achievable expansion features missing in this module. Add them.

TIP: Wait for #31 to avoid rewrite.

$$ expands to os.getpid() and ${!VAR} helps with nesting variables.

Take the example that I got from some random website

VAR1=1
VAR2=2
# var3 is unset.

echo ${VAR1-$VAR2}   # 1
echo ${VAR3-$VAR2}   # 2

This seems to be normal bash behaviour, but in the above example expandvars will output 1 and $VAR2 respectively. Here is the Python code

import os
from expandvars import expandvars

os.environ['VAR1'] = '1'
os.environ['VAR2'] = '2'

v = '${VAR1-$VAR2}'
print(f'{v} expands to "{expandvars(v)}"')    # ${VAR1-$VAR2} expands to "1"

v = '${VAR3-$VAR2}'
print(f'{v} expands to "{expandvars(v)}"')    # ${VAR3-$VAR2} expands to "$VAR2"

This is to continue the discussion on #17 with a broader audience.

I would like to know what other users feel about EXPANDVARS_RECOVER_NULL.

It would be great if %, %%, #, ## expansions were supported
See https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html

Hi,

I have seen that you seem to have released version 0.11.0 of this project. At least the comments in the checkins suggest that.

But the releases 0.10.0 and 0.11.0 don't seem to be tagged in git. This makes it hard for me (and I guess other users too) to download and package these releases in exactly the way they were released.

This is important when you don't wan't to rely on pypi and want to be able to reproduce the builds exactly from source yourself.

Please tag these releases in git. Thank you.

The following example of parameter expansion within another expansion works in bash (Ubuntu and MacOS, also zsh on the latter):

export FAKE_PATH=
echo ${FAKE_PATH:+${FAKE_PATH};}/mongoose
export FAKE_PATH=/wombat
echo ${FAKE_PATH:+${FAKE_PATH};}/mongoose

The intent is to only append the ";" character if "FAKE_BATH" is already set, to avoid getting ";/mongoose" when it isn't. The result is as expected:

/mongoose
/wombat;/mongoose

But Python (3.9) code that should be the equivalent for expandvars:

import expandvars
s= r"${FAKE_PATH:+${FAKE_PATH};}/mongoose"
#print(s)
env = {}
result = expandvars.expand(s, environ=env)
print("%s"%(result))
env["FAKE_PATH"] = "/wombat"
result = expandvars.expand(s, environ=private)
print("%s"%(result))

fails, and produces:

;}/mongoose
${FAKE_PATH;}/mongoose

I'm guessing the expandvars parser can't handle this otherwise bash-acceptable syntax with an expansion inside an expansion?

If you run the example below on Windows you will see that the case-sensitive behaviour changes depending on whether we pass an os._Environ object

import os
from expandvars import expand

d = {'MYVAR': 'blaaaah'}
r = expand('$MyVar', environ=d)
print('With a dict expand is case sensitive...')
print(f'\t$MyVar: {r}')

r = expand('$MYVAR', environ=d)
print(f'\t$MYVAR: {r}')

print('However, using os.environ is not case sensitive')
r = expand('$PROGRAMFILES', environ=os.environ)
print(f'\t$PROGRAMFILES: {r}')

r = expand('$ProgramFiles', environ=os.environ)
print(f'\t$ProgramFiles: {r}')

print('But casting os.environ to a dict makes it case sensitive')
r = expand('$ProgramFiles', environ=dict(os.environ))
print(f'\t$ProgramFiles dict cast: {r}')

${parameter:?word}
If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.

Ref: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html

This module is getting complicated. So far I've avoided using lexers and parsers to keep things simple.
However, I think it's time we dive into these advanced concepts.

TIP: See if we can avoid implementing our own lexer by using regex (I think we can) but without complicating things.

Currently, all the exceptions are being raised as ValueError. It's difficult to distinguish between the errors.

As a better approach raise custom exceptions for each different kind of errors.

Hi, thanks for the library.
I get a weak warning for the code from expandvars import expand - 'expand' is not declared in __all__
Version: expandvars==0.9.0

pip is unable to install from sdist because there is invalid version (0.0.0) in metadata:

$ pip install --no-deps --no-binary :all: expandvars==0.11.0
Defaulting to user installation because normal site-packages is not writeable
Collecting expandvars==0.11.0
  Using cached expandvars-0.11.0.tar.gz (5.9 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
  WARNING: Requested expandvars==0.11.0 from https://files.pythonhosted.org/packages/74/55/7531be1d92ad1a2b1eb59ceaf7871e878db76b55409e68d6e07663a096d2/expandvars-0.11.0.tar.gz, but installing version 0.0.0
Discarding https://files.pythonhosted.org/packages/74/55/7531be1d92ad1a2b1eb59ceaf7871e878db76b55409e68d6e07663a096d2/expandvars-0.11.0.tar.gz (from https://pypi.org/simple/expandvars/) (requires-python:>=3.6): Requested expandvars==0.11.0 from https://files.pythonhosted.org/packages/74/55/7531be1d92ad1a2b1eb59ceaf7871e878db76b55409e68d6e07663a096d2/expandvars-0.11.0.tar.gz has inconsistent version: expected '0.11.0', but metadata has '0.0.0'
ERROR: Ignored the following yanked versions: 0.6.0.macosx-10.15-x86_64
ERROR: Could not find a version that satisfies the requirement expandvars==0.11.0 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.6, 0.2, 0.3, 0.4, 0.4.1, 0.5.0, 0.5.1, 0.5.2, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0)
ERROR: No matching distribution found for expandvars==0.11.0
$

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv expandvars-aVvR3pTZ-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  PackageNotFound

  Package pytest (5.0.1) not found.

  at /usr/local/.pyenv/versions/3.9.0/lib/python3.9/site-packages/poetry/repositories/pool.py:144 in package
      140│                     self._packages.append(package)
      141│ 
      142│                     return package
      143│ 
    → 144│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      145│ 
      146│     def find_packages(
      147│         self, dependency,
      148│     ):

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

This tiny feature is really missing from the original os.path.expendvars and when used with config files that's a big headache.
It'd be nice if you can add support for throwing errors instead of falling to an empty string, a boolean kwarg which can be set and is disabled by default.
I can have a PR ready if you're okay with this.

When expanding long strings with many environment variables, a RecursionError occurs.
See a minimal reproducible error here:

# This causes 'RecursionError: maximum recursion depth exceeded'
import expandvars
long_string = " ".join("$VAR" for _ in range(1000))
expandvars.expand(long_string)

Manually increasing the Python recursion limit can increase how many environment variables can be expanded:

# No exception
import expandvars
import sys
sys.setrecursionlimit(2000)
long_string = " ".join("$VAR" for _ in range(1000))
expandvars.expand(long_string)

It seems like internal expand() and expand_vars() functions call each other recursively, causing the issue.

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv expandvars-66m8TCS8-py3.8 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

[PackageNotFound]
Package tox (3.18.0) not found.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv expandvars--6-FFdBj-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  PackageNotFound

  Package tox (3.20.0) not found.

  at /usr/local/.pyenv/versions/3.9.0/lib/python3.9/site-packages/poetry/repositories/pool.py:144 in package
      140│                     self._packages.append(package)
      141│ 
      142│                     return package
      143│ 
    → 144│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      145│ 
      146│     def find_packages(
      147│         self, dependency,
      148│     ):

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

https://stackoverflow.com/questions/9532654/expression-after-last-specific-character

I quite like this library and want to use it in a project where I provide a conda recipe.
Would you mind publishing the package via conda-forge?
This should be quite easy with Grayskull.

This might be out of scope for this package, but I think it would be very useful if the package could also be to Unix style expression with custom variable characters, for example:

my_secret_access_code = "%{ACCESS_CODE:-default_access_code}"
my_important_variable = "%{IMPORTANT_VARIABLE:?}"
my_updated_path = %PATH:%HOME/.bin"

expandvars(file, varchar="%")

In particular in combination with #25, it would allow to use the package to parse fully custom variable specifications.

The following example correctly displays "$foo" with expandvars 0.7.0, but incorrectly displays None with expandvars 0.8.0:

from expandvars import expandvars
from yaml import load, SafeLoader


foo = '''
path: ${PATH:=bar}
test: \$foo
'''

print(load(expandvars(foo), Loader=SafeLoader))

Example 0.7.0 good output:

{'path': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'test': '$foo'}

Example 0.8.0 bad output:

{'path': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'test': None}

The problem appears to be related to the new recursive calls in expand_advanced.

I have a use case where I would like to use the parser to expand variables in a string with respect to data specified by a custom dict/defaultdict. Something like

expandvars("${ACCESS_CODE:-default_access_code}", source={"ACCESS_CODE": 123})
>>> "123"

This would also allow to only consider a subset of os.environ etc.

filtered_environ = {k: v for k, v in os.environ.items() if k in { "DISPLAY", "NOT_EXISTING"}}
expandvars("$HOME", source=filtered_environ)
>>> ""

The sdist package at PyPI is missing tests. Please add tests to sdist to make downstream testing easier. Thank you.

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv expandvars-CCDwSuZD-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  PackageNotFound

  Package tox (3.14.6) not found.

  at /usr/local/.pyenv/versions/3.9.0/lib/python3.9/site-packages/poetry/repositories/pool.py:144 in package
      140│                     self._packages.append(package)
      141│ 
      142│                     return package
      143│ 
    → 144│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      145│ 
      146│     def find_packages(
      147│         self, dependency,
      148│     ):

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Hi

I tried to install it with pip, but I have the following messages and cannot install it.

pkg_resources.VersionConflict: (importlib-metadata 3.3.0 (/Users/arthuryueh/.local/share/virtualenvs/map-api-QH3cfLJZ/lib/python3.7/site-packages), Requirement.parse('importlib-metadata<3,>=0.12; python_version < "3.8"'))

Please help

Thanks

Builds have started to fail on Python3.3. So let's drop it.

  File "/home/travis/virtualenv/python3.3.6/lib/python3.3/site-packages/pkg_resources/__init__.py", line 92, in <module>
    raise RuntimeError("Python 3.4 or later is required")
RuntimeError: Python 3.4 or later is required

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv expandvars-J7U6z5Iz-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  PackageNotFound

  Package colorama (0.4.1) not found.

  at /usr/local/.pyenv/versions/3.9.0/lib/python3.9/site-packages/poetry/repositories/pool.py:144 in package
      140│                     self._packages.append(package)
      141│ 
      142│                     return package
      143│ 
    → 144│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      145│ 
      146│     def find_packages(
      147│         self, dependency,
      148│     ):

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.