inducer / pymbolic Goto Github PK
View Code? Open in Web Editor NEWA simple package to do symbolic math (focus on code gen and DSLs)
Home Page: http://mathema.tician.de/software/pymbolic
License: Other
A simple package to do symbolic math (focus on code gen and DSLs)
Home Page: http://mathema.tician.de/software/pymbolic
License: Other
Hi,
the example of pymbolic interoperabiity with python ast described in the pymbolic documentation raises an exception under python 3.8.2.
$ python --version
Python 3.8.2
$ cat /tmp/interop_with_python_ast_from_pymbolic_doc.py
src = """
def f():
xx = 3*y + z * (12 if x < 13 else 13)
yy = f(x, y=y)
"""
import ast
mod = ast.parse(src.replace("\n ", "\n"))
print(ast.dump(mod))
from pymbolic.interop.ast import ASTToPymbolic
ast2p = ASTToPymbolic()
for f in mod.body:
if not isinstance(f, ast.FunctionDef):
continue
for stmt in f.body:
if not isinstance(stmt, ast.Assign):
continue
lhs, = stmt.targets
lhs = ast2p(lhs)
rhs = ast2p(stmt.value)
print(lhs, rhs)
$ python /tmp/interop_with_python_ast_from_pymbolic_doc.py
Module(body=[FunctionDef(name='f', args=arguments(posonlyargs=[], args=[], vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), body=[Assign(targets=[Name(id='xx', ctx=Store())], value=BinOp(left=BinOp(left=Constant(value=3, kind=None), op=Mult(), right=Name(id='y', ctx=Load())), op=Add(), right=BinOp(left=Name(id='z', ctx=Load()), op=Mult(), right=IfExp(test=Compare(left=Name(id='x', ctx=Load()), ops=[Lt()], comparators=[Constant(value=13, kind=None)]), body=Constant(value=12, kind=None), orelse=Constant(value=13, kind=None)))), type_comment=None), Assign(targets=[Name(id='yy', ctx=Store())], value=Call(func=Name(id='f', ctx=Load()), args=[Name(id='x', ctx=Load())], keywords=[keyword(arg='y', value=Name(id='y', ctx=Load()))]), type_comment=None)], decorator_list=[], returns=None, type_comment=None)], type_ignores=[])
Traceback (most recent call last):
File "/tmp/interop_with_python_ast_from_pymbolic_doc.py", line 26, in <module>
rhs = ast2p(stmt.value)
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 66, in __call__
return self.rec(expr, *args, **kwargs)
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 80, in rec
return method(self, expr, *args, **kwargs)
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 136, in map_BinOp
return op_constructor(self.rec(expr.left), self.rec(expr.right))
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 80, in rec
return method(self, expr, *args, **kwargs)
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 136, in map_BinOp
return op_constructor(self.rec(expr.left), self.rec(expr.right))
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 82, in rec
return self.not_supported(expr)
File "/home/me/venv38/lib/python3.8/site-packages/pymbolic/interop/ast.py", line 85, in not_supported
raise NotImplementedError(
NotImplementedError: ASTToPymbolic does not know how to map type 'Constant'
FP arithmetic isn't associative, after all.
Minimal reproducer:
In [1]: import pymbolic as p
In [2]: p.parse("True")
Out[2]: Variable('True')
In [3]: p.parse("False")
Out[3]: Variable('False')
The new caching mappers seem to balk at lists (or any unhashable type), since only KeyErrors are caught here.
Actually, there appears to be have already been a "CachingMapperMixin" that handles the corresponding TypeErrors appropriately. Is there a reason for the distinction here or could it be combined with CachedMapper?
For context, I subclass loopy's newly caching-enabled mappers and for convenience enable them to recurse over lists and dicts (usually as the outermost structures).
The implementation in https://github.com/inducer/pymbolic/blob/master/pymbolic/primitives.py#L609 causes collections.Iterable
to consider all pymbolic expressions iterable (although they are explicitly not). I am not sure on the correct way to fix this, as I do not know the reason this error-raising implementation has been added. I am working around this for now, just wanted to let you know.
In [11]: parse("10 if x or y else 20")
raises:
ParseError: else expected, or found instead at index 8: ...or y else 20...
The analogous issue to inducer/pytato#163 also exists in pymbolic.
cc @alexfikl
There was an error encountered while attempting to convert a Pymbolic
floor division
expression to a SymEngine
expression.
import pymbolic.interop.symengine as mapper
import pymbolic.primitives as pr
a = pr.FloorDiv(pr.Sum((pr.Variable('x'),63)),64)
map = mapper.PymbolicToSymEngineMapper()
res = map(a)
print("res: ",res)
print(type(res))
Error:
return self.rec(expr.numerator) // self.rec(expr.denominator)
TypeError: unsupported operand type(s) for //: 'Add' and 'Integer'
I'm currently working on code to perform "delayed operations" on images. After working at it for awhile I'm realizing that what I'm writing is really an expression tree, and what I'm trying to do is simplify that tree before executing it.
A simplified layout of available operations are:
An example tree might look like:
+- Crop
|
+- Cat
|
+- Warp
| |
| +- Cat
| |
| +- Crop
| | |
| | +- Load
| |
| +- Warp
| |
| +- Load
|
+- Warp
|
+- Load
or as a expression C(A(W(A(C(L), W(L))), W(L)))
However, if I was to simply execute this graph it would be inefficient. The crop at the root is likely going to throw away a lot of the intermediate computation. So in practice it might be better to rewrite this tree as:
+- Cat
|
+- Warp
| |
| +- Crop
| |
| +- Load
|
+- Warp
| |
| +- Crop
| |
| + Load
|
+- Warp
|
+- Crop
|
+- Load
and as an expression A(W(C(L)), W(C(L)), W(C(L)))
. (note I'm abusing notation, each operation may have different parameters)
Where all of the cat operations have been moved to the root (because cat is associative A(a, A(b, c)) = A(A(a, b), c)
), all of the warp neighboring warp operations have been squashed together (they are all affine, so just multiply the matrices), and the crops have all moved towards the leafs (For an expression C(W(L))
, we can compute a new crop and warp W' such that C(W(L)) = W'(C'(L))
.
I may have not explained this exactly correctly, but the intuition is that all cats can be delayed until the end, all crops should happen immediately after a load, and all neighboring warps can be combined into a single operation.
In this new simplified expression tree all the crops occur immediately after a load operation, so you are only working with pixels that will influence the final result.
I'm currently slogging through the logic to implement this, but I'm at the point where I'm just going to have to hack it because I don't have a strong grasp on how to implement this properly. I did a bit of searching for resources or packages such that I might be able to define how parameters to operations change when you swap their order in the tree, and then hopefully that package might have a simplify operation that could compute the final structure I'm actually interested in in the general case.
I was wondering if this package might be used for something like that.
https://github.com/inducer/pymbolic/runs/1270178524 shows a failure in the symengine build: IIUC, it seems to want to install the 0.6.1, but then the Python wrapper build looks for 0.6.0...
@isuruf, any tips?
The tests use a python 3.10 syntax and cannot be run on python 3.9.
https://github.com/inducer/pymbolic/blob/main/test/test_pattern_match.py#L33
>>> import pymbolic.primitives as p
>>> a = p.Variable("a")
>>> b = p.Variable("b")
>>> c = p.Variable("c")
>>> print(a * (b * c))
a*b*c # Should have been 'a*(b*c)'
I was playing with the library, and I noticed that expressions seem to require these boilerplate __getinitargs__
and init_arg_names
methods. Given that this is Python 3.6+ it should be easy to automate that via introspection. Here is a proof of concpet:
from pymbolic.primitives import Expression
class AutoInspectable(object):
"""
Helper to provide automatic defaults for pymbolic expressions
"""
def init_arg_names(self):
return tuple(self._initkw().keys())
def __getinitargs__(self):
return tuple(self._initkw().values())
def _initkw(self):
import inspect
from collections import OrderedDict
sig = inspect.signature(self.__class__)
initkw = OrderedDict()
for name, info in sig.parameters.items():
if not hasattr(self, name):
raise NotImplementedError((
'Unable to introspect init args because the class '
'did not have attributes with the same names as the '
'constructor arguments'))
initkw[name] = getattr(self, name)
return initkw
class AutoExpression(AutoInspectable, Expression):
pass
Now classes that inherit from AutoInspectable
will introspect their __init__
functions and as long a there are no *args
**kwargs
and all arguments have instance attributes with the same name, it provides a function to return an "initkw" dictionary (i.e. keyword arguments that you could pass to the class to construct a new instance).
There are edge cases where this doesn't work. Namely *args, **kwargs, and when the instance doesn't exactly register the constructor arguments with the same names, but in those cases it raises a NotImplementedError
indicating that the user can fill these out manually.
Just thought I'd throw that out there.
Not memoizing the results of the graph leads to unnecessary traversals. Some of the expression graphs I have consist of only 25% unique subexpressions i.e. paying unnecessarily for many traversals. I was hoping to implement a memorized version of the mappers (with the id
as a default cheap cache-key). Some questions I had regarding this:
The Mapper
base class suggests pymbolic expression trees should be capable of holding foreign, non-pymbolic objects, at least for numpy arrays because it provides interface for numpy arrays under map_numpy_array
method. However, equaling two expression instances whose children are of foreign, non-hashable types like a numpy array, raises a TypeError
.
When I parse an assignment expression, I am getting a "ParseError: leftover input after completed parse at index 1: ...=x+1...", while it works fine if I parse only the rhs of the assignment. Is this the expected behavior, or am I missing something? Thanks!
In [1]: from pymbolic import parse
...: parse("x+1")
Out[1]: Sum((Variable('x'), 1))
In [2]: parse("y=x+1")
---------------------------------------------------------------------------
ParseError Traceback (most recent call last)
<ipython-input-2-3431e375dd42> in <module>
----> 1 parse("y=x+1")
~/py3env/lib/python3.8/site-packages/pymbolic-2019.2-py3.8.egg/pymbolic/parser.py in __call__(se
lf, expr_str, min_precedence)
536 result = self. parse_expression(pstate, min_precedence)
537 if not pstate.is_at_end():
--> 538 pstate.raise_parse_error("leftover input after completed parse")
539 return result
540
~/py3env/lib/python3.8/site-packages/pytools-2019.1.1-py3.8.egg/pytools/lex.py in raise_parse_er
ror(self, msg)
150 raise ParseError(msg, self.raw_string, None)
151
--> 152 raise ParseError(msg, self.raw_string, self.lexed[self.index])
153
154 def expected(self, what_expected):
ParseError: leftover input after completed parse at index 1: ...=x+1...
When trying to convert a pymbolic expression containing a call to exp (-> Call(Lookup(math), exp)), this line gets called which checks if the Lookup is a prim.Variable. From my limited understanding this will always fail and crash because the Lookup is a Lookup type and not a Variable. Indeed, skipping this check converts normally both to symengine and to sympy. Is this intentional? Has this something to do with the context that should be passed to the SymEngine mapper?
Thank you!
From the documentation:
In prior versions of pymbolic, directly subscripting an Expression subclass generated a Subscript. For various reasons, this was a very bad idea. For example, the following code snippet would result in an infinite loop
If the only issue was the infinite loop, I would argue against the change. It is very nice to be able to write natural subscripts when building expression trees for an embedded DSL that represents arrays.
The infinite loop issue can be prevented by adding the following method to Expression
:
def __iter__(self): raise Exception()
The following test doesn't seem to be making any assertions about expected behavior:
pymbolic/test/test_pymbolic.py
Lines 85 to 94 in 13e83df
Right now, the Sum
and Product
primitives do a little bit of calculation on their own. For example, Sum(<anything>, 0) === <anything>
. Unfortunately, this process is already carried out in parse
so the user can't intervene:
parse(A + 0) = Variable(A)
and parse(A * 1) = Variable(A)
This makes the implicit assumption that 0
is always the identity corresponding to the operator denoted by +
, and that 1
is always the identity corresponding to the operator denoted by *
.
This assumption doesn't always hold (example).
Can you add an option to parse
so that parse(A + 0)
come out to be Sum(Variable('A'), 0)
?
Pymbolic raises an exception when parsing the following expressions:
pymbolic.parse("a >= 1")
pymbolic.parse("a <= 5")
The fix is to change the regexp patterns for 'less' and 'greater' in parser.py so that they do not match "<=" or ">=":
(_less, pytools.lex.RE(r"<[^=]")),
(_greater, pytools.lex.RE(r">[^=]")),
Is there a mapper for listing all subexpressions of an expression? For example, x + 2*y
becomes [x + 2*y, 2*y, x, y]
, sorted based on precedence. Otherwise, if you like the idea, I'm willing to create a PR for it.
Currently, the DifferentiationMapper can handle functions like math.cos or math.log.
import pymbolic as pmbl
x = pmbl.var("x")
u = pmbl.parse("math.cos(x)")
pmbl.differentiate(u, x)
The code block above works and it gives me -sin(x) as a result. However, EvaluationMapper cannot handle these functions. Is there a reason for not having this functionality? Would this be easy to extend on my own?
Maybe antlr?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.