There was an error encountered while attempting to convert a Pymbolic floor division expression to a SymEngine expression.

import pymbolic.interop.symengine as mapper
import pymbolic.primitives as pr

a = pr.FloorDiv(pr.Sum((pr.Variable('x'),63)),64)
map = mapper.PymbolicToSymEngineMapper()
res = map(a)
print("res: ",res)
print(type(res))

Error:

return self.rec(expr.numerator) // self.rec(expr.denominator)
TypeError: unsupported operand type(s) for //: 'Add' and 'Integer'

Pymbolic raises an exception when parsing the following expressions:

pymbolic.parse("a >= 1")
pymbolic.parse("a <= 5")

The fix is to change the regexp patterns for 'less' and 'greater' in parser.py so that they do not match "<=" or ">=":
(_less, pytools.lex.RE(r"<[^=]")),
(_greater, pytools.lex.RE(r">[^=]")),

The new caching mappers seem to balk at lists (or any unhashable type), since only KeyErrors are caught here.

Actually, there appears to be have already been a "CachingMapperMixin" that handles the corresponding TypeErrors appropriately. Is there a reason for the distinction here or could it be combined with CachedMapper?

For context, I subclass loopy's newly caching-enabled mappers and for convenience enable them to recurse over lists and dicts (usually as the outermost structures).

Parser currently can't handle

• bitwise and/or/not/xor
• shifts

@jdsteve2

The following test doesn't seem to be making any assertions about expected behavior:

FP arithmetic isn't associative, after all.

I'm currently working on code to perform "delayed operations" on images. After working at it for awhile I'm realizing that what I'm writing is really an expression tree, and what I'm trying to do is simplify that tree before executing it.

A simplified layout of available operations are:

• L - Load an image from disk
• C - Crop an image to specific bounds
• W - Warp an image via an affine transform
• A - concatenate images along the channel dimension

An example tree might look like:

    +- Crop
    |
    +- Cat
       |
       +- Warp
       |  |
       |  +- Cat
       |     |
       |     +- Crop
       |     |  |
       |     |  +- Load
       |     |
       |     +- Warp
       |        |
       |        +- Load
       |
       +- Warp
           |
           +- Load

or as a expression C(A(W(A(C(L), W(L))), W(L)))

However, if I was to simply execute this graph it would be inefficient. The crop at the root is likely going to throw away a lot of the intermediate computation. So in practice it might be better to rewrite this tree as:

    +- Cat
       |
       +- Warp
       |  |
       |  +- Crop
       |     |
       |      +- Load
       |  
       +- Warp
       |  |
       |  +- Crop 
       |     |
       |     + Load
       |
       +- Warp
           |
           +- Crop
              |
              +- Load

and as an expression A(W(C(L)), W(C(L)), W(C(L))). (note I'm abusing notation, each operation may have different parameters)

Where all of the cat operations have been moved to the root (because cat is associative A(a, A(b, c)) = A(A(a, b), c)), all of the warp neighboring warp operations have been squashed together (they are all affine, so just multiply the matrices), and the crops have all moved towards the leafs (For an expression C(W(L)), we can compute a new crop and warp W' such that C(W(L)) = W'(C'(L)).

I may have not explained this exactly correctly, but the intuition is that all cats can be delayed until the end, all crops should happen immediately after a load, and all neighboring warps can be combined into a single operation.

In this new simplified expression tree all the crops occur immediately after a load operation, so you are only working with pixels that will influence the final result.

I'm currently slogging through the logic to implement this, but I'm at the point where I'm just going to have to hack it because I don't have a strong grasp on how to implement this properly. I did a bit of searching for resources or packages such that I might be able to define how parameters to operations change when you swap their order in the tree, and then hopefully that package might have a simplify operation that could compute the final structure I'm actually interested in in the general case.

I was wondering if this package might be used for something like that.

Maybe antlr?

Minimal reproducer:

In [1]: import pymbolic as p

In [2]: p.parse("True")
Out[2]: Variable('True')

In [3]: p.parse("False")
Out[3]: Variable('False')

Is there a mapper for listing all subexpressions of an expression? For example, x + 2*y becomes [x + 2*y, 2*y, x, y], sorted based on precedence. Otherwise, if you like the idea, I'm willing to create a PR for it.

Right now, the Sum and Product primitives do a little bit of calculation on their own. For example, Sum(<anything>, 0) === <anything>. Unfortunately, this process is already carried out in parse so the user can't intervene:

parse(A + 0) = Variable(A) and parse(A * 1) = Variable(A)

This makes the implicit assumption that 0 is always the identity corresponding to the operator denoted by +, and that 1 is always the identity corresponding to the operator denoted by *.

This assumption doesn't always hold (example).

Can you add an option to parse so that parse(A + 0) come out to be Sum(Variable('A'), 0)?

When trying to convert a pymbolic expression containing a call to exp (-> Call(Lookup(math), exp)), this line gets called which checks if the Lookup is a prim.Variable. From my limited understanding this will always fail and crash because the Lookup is a Lookup type and not a Variable. Indeed, skipping this check converts normally both to symengine and to sympy. Is this intentional? Has this something to do with the context that should be passed to the SymEngine mapper?

Thank you!

The Mapper base class suggests pymbolic expression trees should be capable of holding foreign, non-pymbolic objects, at least for numpy arrays because it provides interface for numpy arrays under map_numpy_array method. However, equaling two expression instances whose children are of foreign, non-hashable types like a numpy array, raises a TypeError.

From the documentation:

In prior versions of pymbolic, directly subscripting an Expression subclass generated a Subscript. For various reasons, this was a very bad idea. For example, the following code snippet would result in an infinite loop

If the only issue was the infinite loop, I would argue against the change. It is very nice to be able to write natural subscripts when building expression trees for an embedded DSL that represents arrays.

The infinite loop issue can be prevented by adding the following method to Expression:

def __iter__(self): raise Exception()

The tests use a python 3.10 syntax and cannot be run on python 3.9.

https://github.com/inducer/pymbolic/blob/main/test/test_pattern_match.py#L33

I was playing with the library, and I noticed that expressions seem to require these boilerplate __getinitargs__ and init_arg_names methods. Given that this is Python 3.6+ it should be easy to automate that via introspection. Here is a proof of concpet:

from pymbolic.primitives import Expression

class AutoInspectable(object):
    """
    Helper to provide automatic defaults for pymbolic expressions
    """

    def init_arg_names(self):
        return tuple(self._initkw().keys())

    def __getinitargs__(self):
        return tuple(self._initkw().values())

    def _initkw(self):
        import inspect
        from collections import OrderedDict
        sig = inspect.signature(self.__class__)
        initkw = OrderedDict()
        for name, info in sig.parameters.items():
            if not hasattr(self, name):
                raise NotImplementedError((
                    'Unable to introspect init args because the class '
                    'did not have attributes with the same names as the '
                    'constructor arguments'))
            initkw[name] = getattr(self, name)
        return initkw


class AutoExpression(AutoInspectable, Expression):
    pass

Now classes that inherit from AutoInspectable will introspect their __init__ functions and as long a there are no *args **kwargs and all arguments have instance attributes with the same name, it provides a function to return an "initkw" dictionary (i.e. keyword arguments that you could pass to the class to construct a new instance).

There are edge cases where this doesn't work. Namely *args, **kwargs, and when the instance doesn't exactly register the constructor arguments with the same names, but in those cases it raises a NotImplementedError indicating that the user can fill these out manually.

Just thought I'd throw that out there.

When I parse an assignment expression, I am getting a "ParseError: leftover input after completed parse at index 1: ...=x+1...", while it works fine if I parse only the rhs of the assignment. Is this the expected behavior, or am I missing something? Thanks!

In [1]: from pymbolic import parse
   ...: parse("x+1")
Out[1]: Sum((Variable('x'), 1))

In [2]: parse("y=x+1")
---------------------------------------------------------------------------
ParseError                                Traceback (most recent call last)
<ipython-input-2-3431e375dd42> in <module>
----> 1 parse("y=x+1")

~/py3env/lib/python3.8/site-packages/pymbolic-2019.2-py3.8.egg/pymbolic/parser.py in __call__(se
lf, expr_str, min_precedence)
    536         result = self. parse_expression(pstate, min_precedence)
    537         if not pstate.is_at_end():
--> 538             pstate.raise_parse_error("leftover input after completed parse")
    539         return result
    540

~/py3env/lib/python3.8/site-packages/pytools-2019.1.1-py3.8.egg/pytools/lex.py in raise_parse_er
ror(self, msg)
    150             raise ParseError(msg, self.raw_string, None)
    151
--> 152         raise ParseError(msg, self.raw_string, self.lexed[self.index])
    153
    154     def expected(self, what_expected):

ParseError: leftover input after completed parse at index 1: ...=x+1...

Currently, the DifferentiationMapper can handle functions like math.cos or math.log.

import pymbolic as pmbl

x = pmbl.var("x")
u = pmbl.parse("math.cos(x)")

pmbl.differentiate(u, x)

The code block above works and it gives me -sin(x) as a result. However, EvaluationMapper cannot handle these functions. Is there a reason for not having this functionality? Would this be easy to extend on my own?

Hi,

the example of pymbolic interoperabiity with python ast described in the pymbolic documentation raises an exception under python 3.8.2.

$ python --version
Python 3.8.2

$ cat /tmp/interop_with_python_ast_from_pymbolic_doc.py
src = """
def f():
    xx = 3*y + z * (12 if x < 13 else 13)
    yy = f(x, y=y)
"""
import ast
mod = ast.parse(src.replace("\n    ", "\n"))

print(ast.dump(mod))

from pymbolic.interop.ast import ASTToPymbolic
ast2p = ASTToPymbolic()

for f in mod.body:
    if not isinstance(f, ast.FunctionDef):
        continue

    for stmt in f.body:
        if not isinstance(stmt, ast.Assign):
            continue

        lhs, = stmt.targets
        lhs = ast2p(lhs)
        rhs = ast2p(stmt.value)

        print(lhs, rhs)

$ python /tmp/interop_with_python_ast_from_pymbolic_doc.py
Module(body=[FunctionDef(name='f', args=arguments(posonlyargs=[], args=[], vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), body=[Assign(targets=[Name(id='xx', ctx=Store())], value=BinOp(left=BinOp(left=Constant(value=3, kind=None), op=Mult(), right=Name(id='y', ctx=Load())), op=Add(), right=BinOp(left=Name(id='z', ctx=Load()), op=Mult(), right=IfExp(test=Compare(left=Name(id='x', ctx=Load()), ops=[Lt()], comparators=[Constant(value=13, kind=None)]), body=Constant(value=12, kind=None), orelse=Constant(value=13, kind=None)))), type_comment=None), Assign(targets=[Name(id='yy', ctx=Store())], value=Call(func=Name(id='f', ctx=Load()), args=[Name(id='x', ctx=Load())], keywords=[keyword(arg='y', value=Name(id='y', ctx=Load()))]), type_comment=None)], decorator_list=[], returns=None, type_comment=None)], type_ignores=[])
Traceback (most recent call last):
  File "/tmp/interop_with_python_ast_from_pymbolic_doc.py", line 26, in <module>
    rhs = ast2p(stmt.value)
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 66, in __call__
    return self.rec(expr, *args, **kwargs)
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 80, in rec
    return method(self, expr, *args, **kwargs)
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 136, in map_BinOp
    return op_constructor(self.rec(expr.left), self.rec(expr.right))
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 80, in rec
    return method(self, expr, *args, **kwargs)
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 136, in map_BinOp
    return op_constructor(self.rec(expr.left), self.rec(expr.right))
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 82, in rec
    return self.not_supported(expr)
  File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 85, in not_supported
    raise NotImplementedError(
NotImplementedError: ASTToPymbolic does not know how to map type 'Constant'

Not memoizing the results of the graph leads to unnecessary traversals. Some of the expression graphs I have consist of only 25% unique subexpressions i.e. paying unnecessarily for many traversals. I was hoping to implement a memorized version of the mappers (with the id as a default cheap cache-key). Some questions I had regarding this:

• Does something like this already exist? (git grep says no)
• Should this be implemented in the base mappers itself? (I'm leaning towards no as memorization wouldn't take into account global state, making it a non-trivial assumption.)

https://github.com/inducer/pymbolic/runs/1270178524 shows a failure in the symengine build: IIUC, it seems to want to install the 0.6.1, but then the Python wrapper build looks for 0.6.0...

@isuruf, any tips?

The analogous issue to inducer/pytato#163 also exists in pymbolic.

cc @alexfikl

In [11]: parse("10 if x or y else 20")

raises:

ParseError: else expected, or found instead at index 8: ...or y else 20...

The implementation in https://github.com/inducer/pymbolic/blob/master/pymbolic/primitives.py#L609 causes collections.Iterable to consider all pymbolic expressions iterable (although they are explicitly not). I am not sure on the correct way to fix this, as I do not know the reason this error-raising implementation has been added. I am working around this for now, just wanted to let you know.