urlquote / f-string issue about django-upgrade HOT 2 CLOSED

Oh fun.

Turns out the tokenize leaves f-strings as a single token:

In [1]: import tokenize_rt

In [2]: tokenize_rt.src_to_tokens('f"{x()}"')
Out[2]:
[Token(name='STRING', src='f"{x()}"', line=1, utf8_byte_offset=0),
 Token(name='NEWLINE', src='', line=1, utf8_byte_offset=8),
 Token(name='ENDMARKER', src='', line=2, utf8_byte_offset=0)]

Whilst the AST parses them, presumably through a second internal tokenization step:

In [3]: import ast

In [4]: print(ast.dump(ast.parse('f"{x()}"'), indent=2))
Module(
  body=[
    Expr(
      value=JoinedStr(
        values=[
          FormattedValue(
            value=Call(
              func=Name(id='x', ctx=Load()),
              args=[],
              keywords=[]),
            conversion=-1)]))],
  type_ignores=[])

The fixer finds the AST Name for the old name, but its callback is then never called because there's no token corresponding to the Name.

I'm not sure what we can do here... perhaps recursively apply the fixers on FormattedValues. Maybe there's something in pyupgrade to copy.

from django-upgrade.

The pyupgrade issue ( asottile/pyupgrade#572 ) was just resolved because it "just works" on Python 3.12. This is thanks to the PEP that modified how f-strings are parsed. I will add a 3.12+ test here to check this is also the case for django-upgrade.

from django-upgrade.