Memory leak related to accumulation of ParseExceptions in `all_exceptions` about moz-sql-parser HOT 3 CLOSED

I may have open this issue prematurely... I have been doing some experimentation no a local clone and I have a better understanding of what is going on.

I tried enabling DEBUG on a local clone and noticed that pyparsing will print all exceptions to stdout. I assume that is why the the pyparsing default is overwritten when DEBUG is false.

I tried the following with DEBUG = False:

if DEBUG:
    debug = (None, None, None)
else:
    debug = (nothing, nothing, nothing)

I got a test failure, and I know that it would break this however it fixed my memory issue:

Dicts and tuples stopped growing. ParseException didn't even register as a common type. The overall memory footprint of my application fell from roughly 1GB per loop iteration to holding steady at 60MB.

It's possible that my use case is unique. I am working on a programming-by-example AI project that synthesizes sql queries. It requires parsing thousands (maybe millions) of generated sql queries, and thus the cost of accumulating the ParseExceptions is high.

I can understand if this is not something you want to officially address in the project, however it would be nice to somehow make record_exception optional.

from moz-sql-parser.

Excellent find! Thank you!

from moz-sql-parser.

Fixed here: 9e97eae?w=1

thank you very much!

from moz-sql-parser.