sum big array causes segmentation fault about numexpr HOT 5 CLOSED

From [email protected] on January 14, 2011 06:06:32

Yes, I can reproduce that (it takes a while, but I see the segfault).

Now that there is preliminary support for the proposed new iterator in NumPy, I was wandering if this would crash too, but we have some problem here:

nesum = ne.evaluate("sum(a, axis=0)")
ValueError: The 'op_axes' provided to the iterator constructor for operand 1 contained duplicate value 0
ne.print_versions()
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version: 2.0.dev
NumPy version: 2.0.0.dev-c44820a
Python version: 2.6.1 ( r261 :67515, Feb 3 2009, 17:34:37)
[GCC 4.3.2 [gcc-4_3-branch revision 141291 ]]
Platform: linux2-x86_64
AMD/Intel CPU? True
VML available? False
Detected cores: 2
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Mark, what do you think?

from numexpr.

From [email protected] on January 24, 2011 11:16:42

The numpy mailing list has some discussion of a problem that sounds related:

numexpr.evaluate gives randomized results on arrays larger than 2047 elements. The following program demonstrates this:

from numpy import *
from numexpr import evaluate

x = zeros(2048)+.01

print evaluate("sum(x, axis = 0)")
print evaluate("sum(x, axis = 0)")

For me this prints different results each time, for example:
11.67
14.84

If we set the size to 2047 I get consistent results.
20.47
20.47

Interestingly, if I do not add .01 to x, it consistently sums to 0.

using numpy 1.5.1 and numexpr 1.4.1

from numexpr.

From [email protected] on January 24, 2011 11:29:31

I should also mention that this was duplicated with prod and at least one other person experienced this with a different array size 8192 (213) instead of 2048 (211).

from numexpr.

From [email protected] on January 25, 2011 02:16:32

Okay. This is a problem with the threading code. Now, forced the use of a single thread on reduction operations. Fixed in r269 .

Status: Verified

from numexpr.

From [email protected] on March 04, 2012 11:27:21

What would be the best way to get the threading code back for sum and prod, while fixing this bug?

from numexpr.