Code Monkey home page Code Monkey logo

desugar's Introduction

desugar

Unravelling Python's syntactic sugar source code.

There are accompanying blog posts to go with all of the code in this repository.

Unravelled syntax

  1. obj.attrbuiltins.getattr(obj, "attr") (including object.__getattribute__())
  2. a + boperator.__add__(a, b)
  3. a - boperator.__sub__(a, b)
  4. a * boperator.__mul__(a, b)
  5. a @ boperator.__matmul__(a, b)
  6. a / boperator.__truediv__(a, b)
  7. a // boperator.__floordiv__(a, b)
  8. a % boperator.__mod__(a, b)
  9. a ** boperator.__pow__(a, b)
  10. a << boperator.__lshift__(a, b)
  11. a >> boperator.__rshift__(a, b)
  12. a & boperator.__and__(a, b)
  13. a ^ boperator.__xor__(a, b)
  14. a | boperator.__or__(a, b)
  15. a += ba = operator.__iadd__(a, b)
  16. a -= ba = operator.__isub__(a, b)
  17. a *= ba = operator.__imul__(a, b)
  18. a @= ba = operator.__imatmul__(a, b)
  19. a /= ba = operator.__itruediv__(a, b)
  20. a //= ba = operator.__ifloordiv__(a, b)
  21. a %= ba = operator.__imod__(a, b)
  22. a **= ba = operator.__ipow__(a, b)
  23. a <<= ba = operator.__ilshift__(a, b)
  24. a >>= ba = operator.__irshift__(a, b)
  25. a &= ba = operator.__iand__(a, b)
  26. a ^= ba = operator.__ixor__(a, b)
  27. a |= ba = operator.__ior__(a, b)
  28. ~ aoperator.__invert__(a)
  29. - aoperator.__neg__(a)
  30. + aoperator.__pos__(a)
  31. a == boperator.__eq__(a, b) (including object.__eq__())
  32. a != boperator.__ne__(a, b) (including object.__ne__())
  33. a < boperator.__lt__(a, b)
  34. a <= boperator.__le__(a, b)
  35. a > boperator.__gt__(a, b)
  36. a >= boperator.__ge__(a, b)
  37. a is boperator.is_(a, b)
  38. a is not boperator.is_not(a, b)
  39. not aoperator.not_(a)
  40. a in boperator.__contains__(b, a)
  41. a not in boperator.not_(operator.__contains__(b, a))
  42. a or b_temp if (_temp := a) else b
  43. a and b_temp if not (_temp := a) else b
  44. import a.ba = __import__('a.b', globals(), locals())
  45. import a.b as cc = __import__('a', globals(), locals(), ['b'], 0).b
  46. from .a import bb = __import__('a', globals(), locals(), ['b'], 1).b
  47. from .a import b as cc = __import__('a', globals(), locals(), ['b'], 1).b
  48. assert ... ➠ see below (post)
  49. for ... ➠ see below (including builtins.iter() and builtins.next())
  50. pass"pass"
  51. with ... ➠ see below (post)
  52. async def ... ➠ see below (post)
  53. await ...desugar.builtins._await(...)
  54. async for ➠ see below (including builtins.aiter() and builtins.anext())
  55. async with ➠ see below (post)
  56. (c for b in a) ➠ see below (post)
  57. [c for b in a]list(c for b in a)
  58. {c for b in a}set(c for b in a)
  59. {c: d for b in a}dict((c, d) for b in a)
  60. [a, b]list((a, b)) (includes iterable unpacking)
  61. {a, b}set((a, b)) (includes iterable unpacking)
  62. (a, b)) ➠ (lambda *args: args)(a, b) (includes iterable unpacking)
  63. {a: b, c: d}) ➠ dict(((a, b), (c, d))) (include dictionary unpacking)
  64. @decorator ➠ see below (post)
  65. break ➠ see below (post)
  66. continue ➠ see below (post)
  67. else clause on while ➠ see below (post)
  68. elif and else clauses on if ➠ see below (post)
  69. else clause on try ➠ see below (post)
  70. finally clause on try ➠ see below (post)
  71. raise A from B ➠ see below (post)
  72. x[A, B]type(x).__getitem__(x, (A, B))
  73. x[A, B] = Ctype(x).__setitem__(x, (A, B), C)
  74. del x[A, B]type(x).__delitem__(x, (A, B))
  75. A:B:Cslice(A, B, C)
  76. 4+3jcomplex(4, 3)
  77. Truebool(1)
  78. Falsebool(0)
  79. None ➠ see below (post)
  80. b"ABC"bytes([65, 66, 67])
  81. "ABC"bytes([65, 66, 67]).decode("utf-8")
  82. ...Ellipsis
  83. class A: ... ➠ see below (post) . ;` ➠ newline plus proper indentation
  84. if ...: ... ➠ see below (post)
  85. a := b see the post
  86. lambda a: b ➠ see below (post)
  87. global A; A = 42getattr(dict, "__setitem__")(globals(), "A", 42)
  88. del A ➠ see below (post)

assert ...

With message

assert a, b

if __debug__:
    if not a:
        raise AssertionError(b)

Without a message

assert a

if __debug__:
    if not a:
        raise AssertionError

for ...

Without else

for a in b:
    c

_iter = iter(b)
while True:
    try:
        a = next(_iter)
    except StopIteration:
        break
    else:
        c
del _iter

With else

for a in b:
    c
else:
    d

_iter = iter(b)
_looping = True
while _looping:
    try:
        a = next(_iter)
    except StopIteration:
        _looping = False
        continue
    else:
        c
else:
    d
del _iter, _looping

with ...

with a as b:
    c

_enter = type(a).__enter__
_exit = type(a).__exit__
b = _enter(a)

try:
    c
except:
    if not _exit(a, *sys.exc_info()):
        raise
else:
    _exit(a, None, None, None)

async def ...

async def spam():
    ...

@types.coroutine
def spam():
    ...

async for ...

Without else

async for a in b:
    c

_iter = aiter(b)
while True:
    try:
        a = await anext(_iter)
    except StopAsyncIteration:
        break
    else:
        c
del _iter

With else

async for a in b:
    c
else:
    d

_iter = aiter(b)
_looping = True
while _looping:
    try:
        a = await anext(_iter)
    except StopAsyncIteration:
        _looping = False
        continue
    else:
        c
else:
    d
del _iter, _looping

async with ...

async with a as b:
    c

_enter = type(a).__aenter__
_exit = type(a).__aexit__
b = await _enter(a)

try:
    c
except:
    if not await _exit(a, *sys.exc_info()):
        raise
else:
    await _exit(a, None, None, None)

(c for b in a)

(c for b in a)

def _gen_exp(_leftmost_iterable):
    for b in _leftmost_iterable:
        yield c

_gen_exp(a)

@decorator

@decorator
def func():
    ...

def func():
    ...

_temp_func_name = func
del func

func = decorator(_temp_func_name)

break

while x:
    break

class _BreakStatement(Exception):
    pass

try:
    while x:
        raise _BreakStatement
except BreakStatement:
    pass

continue

while x:
    continue

class _ContinueStatement(Exception):
    pass

while x:
    try:
        raise _ContinueStatement
    except ContinueStatement:
        pass

else clause on a loop

while x:
    break
else:
    ...

class _BreakStatement(Exception):
    pass

try:
    while x:
        raise _BreakStatement
except _BreakStatement:
    pass
else:
    ...

if

if A:
    B

class _Done(Exception):
    pass

try:
    while A:
        B
        raise _Done
except _Done:
    pass

elif/else on an if statement

if A:
    B
elif C:
    D
else:
    E

_B_ran = _D_ran = False
if A:
    _B_ran = True
    B
if not _B_ran and C:
    _D_ran = True
    D
if not (_B_ran or _D_ran):
    E

try

else

try:
    A
except:
    B
else:
    C

_A_finished = False
try:
    A
    _A_finished = True
except:
    B
if _A_finished:
    C

finally

try:
    A
except Exception:
    B
finally:
    C

try:
    try:
        A
    except Exception:
        B
except BaseException:
    C
    raise
C

raise A from B

raise A from B

_raise = A
if isinstance(_raise, type) and issubclass(_raise, BaseException):
        _raise = _raise()
elif not isinstance(_raise, BaseException):
    raise TypeError("exceptions must derive from BaseException")

_from = B
if isinstance(_from, type) and issubclass(_from, BaseException):
        _from = _from()
if _from is not None:
    _raise.__cause__ = _from

raise _raise

None

None

def _None():
    pass

_None()

class

class Example(SuperClass):
  """Docstring."""
  a: int = 3
  def c(self): return 42

def _exec_Example(_ns):
    _temp_ns = {}

    _temp_ns["__module__"] = _ns["__module__"] = __name__
    _temp_ns[__"qualname__"] = _ns["__qualname__"] = "Example"
    _temp_ns["__doc__"] = _ns["__doc__"] = """Docstring."""
    _temp_ns["__annotations__"] = _ns["__annotations__"] = {"a": int}

    _temp_ns["a"] = _ns["a"] = 3

    def _method_c(self):
        return 42
    _method_c.__name__ = "c"
    _method_c.__qualname__ = "Example.c"
    temp_ns["c"] = _ns["c"] = _method_c
    del _method_c

def _class_Example():
    # Resolving MRO entries.
    bases = types.resolve_bases((SuperClass, ))
    # Determining the appropriate metaclass **and**
    # preparing the class namespace.
    meta, ns, kwds = types.prepare_class("Example", bases)
    # Executing the class body.
    _exec_Example(ns)
    # Creating the class object.
    cls = meta("Example", bases, ns)
    ## Class decorators, if there were any.
    ## Make the namespace read-only.
    cls.__dict__ = read_only_proxy(ns)

    return cls

 Example = _class_Example()

lambda

lambda A: B

def _lambda(A):
    return B
_lambda.__name__ = "<lambda>"

del

del A

Local

_DELETED = object()

# `del A`
A = _DELETED
# Referencing `A`
if A is _DELETED:
    raise UnboundLocalError("cannot access local variable 'A' where it is not associated with a value")

Global

try:
    gettattr(globals(), "__delitem__")("A")
except KeyError:
    raise NameError("name 'A' is not defined")

Syntax that can't be unravelled

Summary post

Keywords

Taken from the keyword module.

Nothing; all unravelled!

Expressions

  1. yield

Statements

  1. def

  2. nonlocal

  3. return

  4. try/except

  5. while

Tokens

Taken from the token module.

  1. =

  2. () for calls

  3. ,

Literals

  1. Integers

desugar's People

Contributors

brettcannon avatar geryogam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

desugar's Issues

Should decorators remain sugary?

There is likely a rationale that escapes me, but I noticed decorators were not mentioned in the README. It's the first thing that comes to mind when I think of syntactic sugar. Though, I realize the @ symbol here may not quite fit in your categories. Are decorators intentionally excluded?

Also, are you excluding with/as and except/as?

Btw, I'm enjoying the blogs. Thanks.

Unravelling `finally` from `try` statements

In https://snarky.ca/unravelling-finally-and-else-from-try/ you wrote:

try:
    A
except Exception:
    B
finally:
    C

Example try statement with a finally clause
we can transform it into:

try:
    try:
        A
    except Exception:
        B
except BaseException:
    C
    raise
C

Could it be that this transformation is not entirely correct?

Given:

def f(n):
    try:
        return 1/n
    except Exception:
        print('except')
    finally:
        print('finally')

Calling this function gives the following results depending whether an exception was raised or not:

>>> print(f(1))
finally
1.0
>>> print(f(0))
except
finally
None

Transforming the function as suggested in the blog post I get:

def g(n):
    try:
        try:
            return 1/n
        except Exception:
            print('except')
    except BaseException:
        print('finally')
        raise
    print('finally')

Calling this function I get different results:

>>> print(g(1))
1.0
>>> print(g(0))
except
finally
None

So in case no exception is raised the finally part of the transformed function is not called.

Am I missing something here or did I make a mistake in the transformation?

Errors in "Unravelling `elif`/`else` from `if` statements" post

I'm referring to this post - couldn't find the actual source for the post (and the error don't show up in this repo), so sorry if this is not the right place to report :-)

towards the end, the last example is:

if a:
    b
elif c:
    d
elif e:
    f
else:
    g

which should become:

_b_ran = _d_ran = _f_ran = False
if a:
    _b_ran = True
    b
if not _b_ran and c:
    _d_ran = True
    d
if not (_b_ran or _d_ran) and e:
    _f_ran = True
    f
if not (_b_ran or _d_ran or _f_ran):
    g

as opposed to what's currently on the post:

_b_ran = _d_ran = _f_ran = False
if a:
    _b_ran = True
    b
if not _b_ran and c:
    _d_ran = True
if not (_b_ran or _d_ran) and e:
    _g_ran = True
if not (_b_ran or _d_ran or _f_ran):
    g

Thanks for writing this series - it's super interesting and informative!

The Python implementation of object.__getattribute__ does not detect descriptors correctly

From the Python implementation of object.__getattribute__ in your blog article Unravelling attribute access in Python:

class object:
    def __getattribute__(self: Any, attr: str) -> Any:
        # […]
        for base in self_type.mro():
            if attr in base.__dict__:
                type_attr = base.__dict__[attr]
                type_attr_type = type(type_attr)
                # MISSING FOR-LOOP: for attr_base in type_attr_type.mro():
                if "__get__" in type_attr_type.__dict__:
                    descriptor_type_get = type_attr_type.__dict__["__get__"]
                    # Include/descrobject.h:PyDescr_IsData
                    if "__set__" in type_attr_type.__dict__:
                        # Data descriptor.
                        return descriptor_type_get(type_attr, self, self_type)
                    else:
                        break  # Non-data descriptor.
                else:
                    break  # Plain object.
        # […]

So you are correctly looking up the attribute named attr in the types of the mro() of self_type. However you are not looking up the method __get__ in all the types of the mro() of the found attribute’s type but only in the found attribute’s type. In other words, you are ignoring the base types of the found attribute’s type.

Consequently, your Python implementation is failing to behave as expected below by considering foo as a class attribute instead of a non-data descriptor:

>>> class A:
...     def __get__(self, instance, owner):
...         return 'bar'
... 
>>> class B(A):
...     pass
... 
>>> class C:
...     foo = B()
... 
>>> C().foo
'bar'  # Your implementation returns '<__main__.B object at 0x101d26d60>' instead.

To fix this you should also loop on type_attr_type.mro() in your Python implementation (cf. my comment MISSING FOR-LOOP above).

Binary arithmetic dispatch is not implemented correctly

With the following classes

class Foo:
    def __sub__(self, a):
        return ('sub', self, a)
    def __rsub__(self, a):
        return ('rsub', self, a)

class Bar(Foo):
    pass

desugar and CPython disagree on the result:

In [175]: from desugar.operator import sub

In [176]: sub(Foo(), Bar())
Out[176]: ('rsub', <__main__.Bar at 0x1f380795220>, <__main__.Foo at 0x1f380795b50>)

In [177]: Foo() - Bar()
Out[177]: ('sub', <__main__.Foo at 0x1f3804c6f40>, <__main__.Bar at 0x1f3804c6d00>)

See https://bugs.python.org/issue30140 for details - the conclusion was

Let's give up on this one.

Which led to the PR at python/cpython#1325 being closed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.