Code Monkey home page Code Monkey logo

Comments (3)

CensoredUsername avatar CensoredUsername commented on August 9, 2024 1

How can we differentiate docstrings and multiline strings? / How do we already do this? (if we do it at all)

You can't, because docstrings are literally just multiline strings. There's no reason to differentiate between them.

Why is there a match against an empty string for a triple quote in the Python RegEx? If we just never try match this, it should be added in the comment above the RegEx.

That's just the result of the regex not recognizing the mutliline string, and thus it falls back on matching the initial two quotes as an empty string.

The issue exists because you're not compiling the regex with the same options that unrpyc does. the python code ends up running that regex with re.DOTALL enabled, which means . also matches newlines. Here's it with that option enabled.

Where is this PEP 257 Implementation done regarding multiline docstrings? Is it implicit somewhere? If so, it should also be added in comments

Docstrings are literally just normal python strings.

Now there's also a general note here. This function isn't trying to be absolutely correct. It's trying to do what ren'py does here. For the purpose of checking that we aren't generating output that ren'py would fail on. If it fails to parse something that has become legal, we'll just put some parenthesis around it and be done with it (see simple_expression_guard).

(Now that function has had some recent changes, like accepting [urfURF] as allowed prefixes, but as I said before, false negatives are okay here, and we can fix that when we get there).

from unrpyc.

madeddy avatar madeddy commented on August 9, 2024

Hi. Two things come to mind:

  • maybe add to which code/func/module in unrpyc this refers (citing.. link)
  • afaik multiline strings have in py special rules regarding indent and newline preservation. Maybe this influences your results and i think there was a PEP for this.

from unrpyc.

Lumpy-dev avatar Lumpy-dev commented on August 9, 2024

Here's the function I am refeering to:

unrpyc/decompiler/util.py

Lines 390 to 397 in 79895a1

def python_string(self, clear_whitespace=True):
# parse strings the ren'py way (don't parse docstrings, no b/r in front allowed)
# edit: now parses docstrings correctly. There was a degenerate case where '''string'string''' would
# result in issues
if clear_whitespace:
return self.match(r"""(u?(?P<a>"(?:"")?|'(?:'')?).*?(?<=[^\\])(?:\\\\)*(?P=a))""")
else:
return self.re(r"""(u?(?P<a>"(?:"")?|'(?:'')?).*?(?<=[^\\])(?:\\\\)*(?P=a))""")

I want to add the fact that the comment talks about not parsing docstrings.

But this makes my think of more issues:

  • How can we differentiate docstrings and multiline strings? / How do we already do this? (if we do it at all)
  • Why is there a match against an empty string for a triple quote in the Python RegEx? If we just never try match this, it should be added in the comment above the RegEx.
  • Where is this PEP 257 Implementation done regarding multiline docstrings? Is it implicit somewhere? If so, it should also be added in comments

from unrpyc.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.