Optimization of "compilation phase" through a better hash about numexpr HOT 5 CLOSED

From [email protected] on January 24, 2011 00:24:14

Okay. I'll have a look at your patch. Could you please send a small benchmark showing the speed-ups of your approach?

Thanks!

from numexpr.

From [email protected] on January 24, 2011 01:55:45

Here is a modified version of bench/varying-expr.py which shows a nice speedup:
on my machine it goes from 17.596 seconds to 10.799 seconds with the patch.

Of note is that the unmodified varying-expr.py does not show any improvement. I guess that expression is too simple to trigger the repetitive hash calls.

Again, this is probably more of a proof of concept than a real patch, because it suppose ASTNode is read-only (which seem to be the case) but is not enforced anywhere and it would cause very weird errors if the nodes are ever modified.

Attachment: latency.py

from numexpr.

From [email protected] on January 24, 2011 08:39:38

As I feared, the ASTNode were not really read-only. They are at least modified in collapseDuplicateSubtrees. I don't think that caching patch introduced any problem anyway because of the way the hash is used there, but it's too risky.

FWIW, the other patch yields "14.38 seconds" to the "latency.py" test.

from numexpr.

From [email protected] on January 25, 2011 04:42:36

I'm not familiar with that part of the code, so let's walk on the safe side. So will apply the first patch (done in #273).

Anyway, if in the future you can demonstrate that the second patch is safe for any practical use, I'll be glad to get it in.

Status: Verified

from numexpr.

From [email protected] on January 25, 2011 05:07:45

Well, this seems risky stuff and doing it in a way that is both correct and faster would probably take more time than I'm willing to invest for such minor gains.

from numexpr.