numba / numba Goto Github PK

View Code? Open in Web Editor NEW

9.5K 202.0 1.1K 76.65 MB

NumPy aware dynamic Python compiler using LLVM

Home Page: http://numba.pydata.org/

License: BSD 2-Clause "Simplified" License

Python 90.95% Shell 0.14% C 6.37% C++ 2.47% HTML 0.03% Cuda 0.03% Batchfile 0.02% GDB 0.01%

python numpy llvm compiler cuda parallel

numba's Introduction

Numba

A Just-In-Time Compiler for Numerical Functions in Python

Numba is an open source, NumPy-aware optimizing compiler for Python sponsored by Anaconda, Inc. It uses the LLVM compiler project to generate machine code from Python syntax.

Numba can compile a large subset of numerically-focused Python, including many NumPy functions. Additionally, Numba has support for automatic parallelization of loops, generation of GPU-accelerated code, and creation of ufuncs and C callbacks.

For more information about Numba, see the Numba homepage: https://numba.pydata.org and the online documentation: https://numba.readthedocs.io/en/stable/index.html

Installation

Please follow the instructions:

https://numba.readthedocs.io/en/stable/user/installing.html

Demo

Please have a look and the demo notebooks via the mybinder service:

https://mybinder.org/v2/gh/numba/numba-examples/master?filepath=notebooks

Contact

Numba has a discourse forum for discussions:

https://numba.discourse.group

numba's People

Contributors

Stargazers

Watchers

Forkers

ogrisel andreacrotti cwgreene aterrel npinto prafulm afrolov jriehl bluemoon jaberg markflorisson certik sklam teoliphant twiecki mszep nfoti yamins81 astrojuanlu rlamy ericjustusson dmcdougall seanjensengrey takluyver zed meteogrid aseyboldt yarikoptic buguen yuchao86 renjiec glycerine mekman kanghaiyang chiehwen jdchristensen cu6yu4 hfeeki larsmans hgrecco mspacek lizecillie dwf garrison mdavidovic maksim2042 shiquanwang tree-wizard sprig gdementen jgors cooperliu101 asmeurer nouiz ejmvar magicbill martinfiers bfredl kichik iconica majidaldo tpn francescalted pombredanne sergiopasra nvdnkpr skariel funningboy jenstimmerman azizur77 dichaelen bruce2008github tillsten svaksha chtlp zzmjohn desireb laserson aburan28 rayleyva apsaltis jenshnielsen ewiger albop tempbottle wojons neeck agamdua falcondai invinciblejha pitrou frenchrd mjjoyce winstonewert xiaweiwang esc prodigeni jiaxu825 whalen53 samkohli

numba's Issues

Casting is not supported

At the moment, it is not possible to cast numerical values: e.g. Int to float.
As a interim solution one can use 1.0*int_val.

llvm::Module goes out of scope in vectorize decorator.

The vectorize decorator builds an LLVM Module and ExecutionEngine, and gets a pointer from the ExecutionEngine. However, these go out of scope and are garbage collected when the decorator is finished. This can cause the function pointer for the ufunc to map to invalid memory causing a segmentation fault.

I am able to verify this in gdb by putting breakpoints on ufunc_from_ptr, llvm::Module::~Module, and MyUFunc_D_D, and running code that uses the decorator in module initialization. The important point is that the breakpoint for llvm::Module::~Module is hit before the call to MyUFunc_D_D, where the segfault occurs.

Note that I intend to fix this, but I'm documenting this so I don't forget this between development sessions.

Numpy complaining of reference count error in test suite.

When I run the test suite I'm (almost) consistently seeing the following output:

(numba)jriehl@dingo:~/git/numba/tests$ python -O test_all.py 
...
----------------------------------------------------------------------
Ran 33 tests in 0.369s

OK (skipped=1)
*** Reference count error detected: 
an attempt was made to deallocate 12 (d) ***
(numba)jriehl@dingo:~/git/numba/tests$

I'm pleased to see 32 of 33 tests pass, but the reported reference count error indicates we aren't doing something correctly.

On the surface, this looks very much like a problem reported on Numpy-discussion here: http://mail.scipy.org/pipermail/numpy-discussion/2012-January/059866.html

That discussion identifies the Numpy C API where this error message is output.

The reference count error started happening in test_all.py runs when I got the filter2d() test passing (see .../tests/test_filter2d.py; note that running this test alone doesn't seem to cause this problem). If this was a problem with reference counts on the result of a call to PyArray_Zeros(), I would have thought it would have shown up before now (by way of .../tests/test_extern_call.py). I can try adding a call to Py_IncRef() in the generated code and see if that fixes it, but I'm not sure that just won't introduce a memory leak. I understand we don't care too much about memory leaks in the compiler at the moment, but leaks in generated code would be much more serious.

Example crashes on Mac OS X

I am running

Mac OS 10.7.3
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
numpy '2.0.0.dev-26aa3cf'
scipy '0.10.0.dev'

I installed LLVM, llvm-py and numba as explained in the readme. On running the example script, I get this error:

Assertion failed: ((i >= FTy->getNumParams() || FTy->getParamType(i) == Args[i]->getType()) && "Calling a function with a bad signature!"), function init, file Instructions.cpp, line 274.
Abort trap: 6

Broken inference of call to (x)range

See

https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/co6bDzfHsBw

Returning borrowed reference Python objects causes reference count error.

Making a small modification to the fbcorr code like this, to call the fbcorr() method from within a for loop causes a crash:

import numpy as np

from numba.decorators import jit
nd4type = [[[['d']]]]

@jit(ret_type=nd4type, arg_types=(nd4type, nd4type, nd4type))
def fbcorr(imgs, filters, output):
    n_imgs, n_rows, n_cols, n_channels = imgs.shape
    n_filters, height, width, n_ch2 = filters.shape

    for ii in range(n_imgs):
        for rr in range(n_rows - height + 1):
            for cc in range(n_cols - width + 1):
                for hh in xrange(height):
                    for ww in xrange(width):
                        for jj in range(n_channels):
                            for ff in range(n_filters):
                                imgval = imgs[ii, rr + hh, cc + ww, jj]
                                filterval = filters[ff, hh, ww, jj]
                                output[ii, ff, rr, cc] += imgval * filterval

    return output

def main ():
    imgs = np.random.randn(10, 64, 64, 3)
    filt = np.random.randn(6, 5, 5, 3)
    output = np.zeros((10, 60, 60, 6))

    import time
    t0 = time.time()
    for i in range(5):
        fbcorr(imgs, filt, output)
    print time.time() - t0

Produces the following crash:

In [3]: main()
python(59743,0x7fff7bf5d960) malloc: *** error for object 0x10dcd5000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
[1]    59743 abort      ipython

Double free error

When I tried to accelerated lagrange polynomials calculation (toy example) I got double free error:

@jit(restype=double[:,:], argtypes=[double[:], double[:,:]])
def lagrange_base_(points, out):
    "Compute lagrange polynomial."
    k = points.shape[0]
    #out = np.empty((k, k))
    #assert out.shape == (k, k)
    #assert points.shape == (k,)
    for j in range(k):
        l_j = out[j]
        l_j[-1] = 1
        for m in range(k):
            if m != j:
                #m_init = m if m > j else m + 1
                norm = 1/(points[j] - points[m])
                for i in range(k-1):
                    l_j[i] = norm * l_j[i + 1] - norm * points[m] * l_j[i]
                l_j[k - 1] *= -norm * points[m]

No symbol "all" in current context.
#0  0x00007ffff748deb5 in __GI_raise (sig=<optimized out>)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:63
        resultvar = 0
        pid = <optimized out>
        selftid = <optimized out>
#1  0x00007ffff748f32b in __GI_abort () at abort.c:90
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0x4, sa_sigaction = 0x4}, 
          sa_mask = {__val = {5, 140737488347291, 16, 140737343387832, 3, 
              140737488342410, 6, 140737343387836, 2, 140737488342430, 2, 
              140737343378913, 1, 140737343387832, 3, 140737488342404}}, 
          sa_flags = 12, sa_restorer = 0x7ffff75bf8bc}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007ffff74cda4e in __libc_message (do_abort=2, 
    fmt=0x7ffff75c1458 "*** glibc detected *** %s: %s: 0x%s ***\n")
    at ../sysdeps/unix/sysv/linux/libc_fatal.c:197
        ap = {{gp_offset = 40, fp_offset = 48, 
            overflow_arg_area = 0x7fffffffd780, 
            reg_save_area = 0x7fffffffd690}}
        ap_copy = {{gp_offset = 16, fp_offset = 48, 
            overflow_arg_area = 0x7fffffffd780, 
            reg_save_area = 0x7fffffffd690}}
        fd = 7
        on_2 = <optimized out>
        list = <optimized out>
        nlist = <optimized out>
        cp = <optimized out>
        written = <optimized out>
#3  0x00007ffff74d3746 in malloc_printerr (action=3, 
    str=0x7ffff75c15b8 "double free or corruption (out)", ptr=<optimized out>)
    at malloc.c:4949
        buf = "00000000014c4560"
        cp = <optimized out>
#4  0x00007ffff073b2f2 in array_dealloc (self=0x15402d0)
    at numpy/core/src/multiarray/arrayobject.c:267
No locals.
#5  0x00007ffff7a9b067 in insertdict (mp=0x640f80, key=0x7ffff7f76210, 
    hash=8448025411, value=0x7ffff7da4ea0 <_Py_NoneStruct>)
    at Objects/dictobject.c:530
        old_value = <optimized out>
        ep = 0xc377b8
#6  0x00007ffff7a9d6ee in PyDict_SetItem (op=0x640f80, key=<optimized out>, 
    value=0x7ffff7da4ea0 <_Py_NoneStruct>) at Objects/dictobject.c:775
        mp = 0x640f80
        hash = <optimized out>
        n_used = 14
#7  0x00007ffff7aa1172 in _PyModule_Clear (m=<optimized out>)
    at Objects/moduleobject.c:138
        s = <optimized out>
        pos = 4
        key = 0x7ffff7f76210
        value = 0x15402d0
        d = 0x640f80
#8  0x00007ffff7b0e27d in PyImport_Cleanup () at Python/import.c:445
        pos = <optimized out>
        ndone = <optimized out>
        name = <optimized out>
        key = <optimized out>
        value = 0x7ffff7fb4be8
        dict = <optimized out>
        interp = 0x602010
        modules = 0x620150
#9  0x00007ffff7b1a99a in Py_Finalize () at Python/pythonrun.c:454
        interp = 0x602010
#10 Py_Finalize () at Python/pythonrun.c:400
No locals.
#11 0x00007ffff7b199e8 in Py_Exit (sts=0) at Python/pythonrun.c:1760
No locals.
#12 0x00007ffff7b19b14 in handle_system_exit () at Python/pythonrun.c:1134
        exception = 0x7ffff7d97b80 <_PyExc_SystemExit>
        value = 0x61e7d0
        tb = 0x13f9fc8
        exitcode = 0
#13 0x00007ffff7b19db5 in handle_system_exit () at Python/pythonrun.c:1090
        exitcode = 0
#14 PyErr_PrintEx (set_sys_last_vars=1) at Python/pythonrun.c:1144
        exception = <optimized out>
        v = <optimized out>
        tb = <optimized out>
        hook = <optimized out>
#15 0x00007ffff7b1a233 in PyRun_SimpleFileExFlags (fp=<optimized out>, 
    filename=<optimized out>, closeit=1, flags=0x7fffffffdb80)
    at Python/pythonrun.c:947
        m = <optimized out>
        d = 0x640f80
        v = <optimized out>
        ext = <optimized out>
        set_file_name = 1
        ret = <optimized out>
        len = <optimized out>
#16 0x00007ffff7b2b904 in Py_Main (argc=<optimized out>, argv=<optimized out>)
    at Modules/main.c:643
        c = <optimized out>
        sts = -1
        command = 0x0
        filename = 0x7fffffffe0ac "XXXXXX"
        module = 0x0
        fp = 0x6c5650
        p = <optimized out>
        unbuffered = <optimized out>
        skipfirstline = 0
        stdin_is_interactive = 1
        help = <optimized out>
        version = <optimized out>
        saw_unbuffered_flag = -8020
        cf = {cf_flags = 0}
        target_script_name = 0x1 <Address 0x1 out of bounds>
#17 0x00007ffff747a82d in __libc_start_main (main=0x400880 <main>, argc=2, 
    ubp_av=0x7fffffffdca8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffdc98) at libc-start.c:225
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -1256593659022039821, 
                4196596, 140737488346272, 0, 0, 1256593658088236275, 
                1256574899949642995}, mask_was_saved = 0}}, priv = {pad = {
              0x0, 0x0, 0x4009e0 <__libc_csu_init>, 0x7fffffffdca8}, data = {
              prev = 0x0, cleanup = 0x0, canceltype = 4196832}}}
        not_first_call = <optimized out>
#18 0x000000000040091d in _start ()
No symbol table info available.
A debugging session is active.

argument-free functions require an argument after compilation

Standard Python:

def no_args_method():
    return 3.0
print no_args_method()

JIT'd Numba:

@jit(ret_type=numba.double)
def no_args_method():
    return 3.0
print no_args_method()

The error on the Numba code:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
----> 1 print no_args_method()

TypeError: this function takes at least 1 argument (0 given)

A weird and (mostly) unimportant bug.

Test errors with Macports llvm 3.1

I have installed the latest llvmpy e0d1b94053517bccb30f88b53d541cd4b986dfbb and numba 5a92630 using Macports llvm-3.1 @3.1_4 on Mac OS X 10.8.

The llvmpy build gives these warnings:

/usr/bin/clang++ -bundle -undefined dynamic_lookup -L/opt/local/lib build/temp.macosx-10.8-x86_64-2.7/llvm/_core.o build/temp.macosx-10.8-x86_64-2.7/llvm/wrap.o build/temp.macosx-10.8-x86_64-2.7/llvm/extra.o -L/opt/local/libexec/llvm-3.1/lib -lLLVMPTXCodeGen -lLLVMPTXDesc -lLLVMPTXInfo -lLLVMPTXAsmPrinter -lLLVMLinker -lLLVMArchive -lLLVMAsmParser -lLLVMipo -lLLVMVectorize -lLLVMInstrumentation -lLLVMBitWriter -lLLVMBitReader -lLLVMInterpreter -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMMCParser -lLLVMX86Disassembler -lLLVMX86Desc -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMJIT -lLLVMRuntimeDyld -lLLVMCodeGen -lLLVMExecutionEngine -lLLVMScalarOpts -lLLVMInstCombine -lLLVMTransformUtils -lLLVMipa -lLLVMAnalysis -lLLVMTarget -lLLVMMC -lLLVMObject -lLLVMCore -lLLVMSupport -o build/lib.macosx-10.8-x86_64-2.7/llvm/_core.so -L/opt/local/libexec/llvm-3.1/lib -lpthread -lffi -lm
ld: warning: direct access in llvm::fouts() to global weak symbol llvm::formatted_raw_ostream::~formatted_raw_ostream() means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in llvm::ferrs() to global weak symbol llvm::formatted_raw_ostream::~formatted_raw_ostream() means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in llvm::fdbgs() to global weak symbol llvm::formatted_raw_ostream::~formatted_raw_ostream() means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.

The llvmpy test runs without errors.

The numba import gives this warning:

In [1]: import numba

WARNING -- functions:17:<module>
Could not import Meta - AST translation will not work:
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 14, in <module>
    from meta.decompiler import decompile_func
ImportError: No module named meta.decompiler

Most numba tests fail:

In [2]: numba.test()

WARNING -- functions:17:<module>
Could not import Meta - AST translation will not work:
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 14, in <module>
    from meta.decompiler import decompile_func
ImportError: No module named meta.decompiler

EsEEEEEEEEEEsE.s....EEE.2
2
3
.E.s......
WARNING -- decorators:232:_jit
Warning: Previously compiled version of <function get_ndarray_data at 0x110e8a140> may be garbage collected!
..
WARNING -- decorators:232:_jit
Warning: Previously compiled version of <function get_ndarray_ndim at 0x110e8a050> may be garbage collected!
..
WARNING -- decorators:232:_jit
Warning: Previously compiled version of <function get_ndarray_shape at 0x110e8a0c8> may be garbage collected!
............E.E..............
======================================================================
ERROR: test_numba (numba.tests.test_ast_arrays.TestASTArrays)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_ast_arrays.py", line 36, in test_numba
    matmulcore(A, B, C)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 169, in wrapper
    context, f, argtypes=types)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_call_len (numba.tests.test_extern_call.TestASTExternCall)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_extern_call.py", line 52, in test_call_len
    call_len)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_call_zeros_like (numba.tests.test_extern_call.TestASTExternCall)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_extern_call.py", line 42, in test_call_zeros_like
    call_zeros_like)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_numba_calls_numba (numba.tests.test_extern_call.TestASTExternCall)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_extern_call.py", line 56, in test_numba_calls_numba
    self.assertEqual(func2(3), 8)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_0 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 134, in test_while_loop_fn_0
    test_data)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 128, in _do_test
    compiled_fn = _jit(function)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_1 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 137, in test_while_loop_fn_1
    self._do_test(while_loop_fn_1, [double[:]], numpy.array([1., 2., 3.]))
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 128, in _do_test
    compiled_fn = _jit(function)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_2 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 140, in test_while_loop_fn_2
    self._do_test(while_loop_fn_2, [double[:]], numpy.array([1., 2., 3.]))
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 128, in _do_test
    compiled_fn = _jit(function)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_3 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 143, in test_while_loop_fn_3
    compiled_fn = self.jit(argtypes = [long_])(while_loop_fn_3)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_4 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 150, in test_while_loop_fn_4
    restype = long_)(while_loop_fn_4)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_5 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 156, in test_while_loop_fn_5
    compiled_fn = self.jit(argtypes = [double, double])(while_loop_fn_5)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_6 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 162, in test_while_loop_fn_6
    compiled_fn = self.jit(restype=double, argtypes=[double])(while_loop_fn_6)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_while_loop_fn_8 (numba.tests.test_while.TestASTWhile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_while.py", line 172, in test_while_loop_fn_8
    compiled_fn = self.jit(restype=double, argtypes=[double])(while_loop_fn_8)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 245, in _jit
    return jit2(argtypes=argtypes)(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_conversion (numba.tests.test_object_conversion.TestConversion)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_object_conversion.py", line 31, in test_conversion
    assert convert(object(), 10.2) == 10.2
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_numeric_conversion (numba.tests.test_object_conversion.TestConversion)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_object_conversion.py", line 68, in test_numeric_conversion
    result = convert_numeric(value, dst_type)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_pointer_conversion (numba.tests.test_object_conversion.TestConversion)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_object_conversion.py", line 75, in test_pointer_conversion
    result = convert_to_pointer(array)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_numba_calls_numba (numba.tests.test_extern_call.TestExternCall)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_extern_call.py", line 56, in test_numba_calls_numba
    self.assertEqual(func2(3), 8)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_mandel (numba.tests.test_mandelbrot.TestMandelbrot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_mandelbrot.py", line 93, in test_mandel
    self.assertEqual(mandel(real, imag, 20),
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_mandel_driver (numba.tests.test_mandelbrot.TestMandelbrot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_mandelbrot.py", line 99, in test_mandel_driver
    mandel_driver.py_func(-1., 1., -1., len(palette), palette, control_image)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/tests/test_mandelbrot.py", line 56, in mandel_driver
    color = mandel(real, imag, nb_iterations)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 119, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 140, in wrapper
    compiled_numba_func = dec(f)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 216, in _jit2_decorator
    **kwargs)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 128, in compile_function
    ctypes=ctypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 62, in _compile
    restype, argtypes, **kwds)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 46, in _infer_types
    ast = _get_ast(func)
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 41, in _get_ast
    return decompile_func(func)
TypeError: 'NoneType' object is not callable

----------------------------------------------------------------------
Ran 67 tests in 0.278s

FAILED (errors=18, skipped=4)
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
<ipython-input-2-7e0c52f6aaa7> in <module>()
----> 1 numba.test()

/Users/deil/Library/Python/2.7/lib/python/site-packages/numba/__init__.pyc in test()
     24     from subprocess import check_call
     25 
---> 26     check_call([sys.executable, '-m', 'numba.tests.test_all'])
     27 
     28 

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.pyc in check_call(*popenargs, **kwargs)
    509         if cmd is None:
    510             cmd = popenargs[0]
--> 511         raise CalledProcessError(retcode, cmd)
    512     return 0
    513 

CalledProcessError: Command '['/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', '-m', 'numba.tests.test_all']' returned non-zero exit status 1

DeferredCoercionNode init conflict with new in CoercionNode

This problem occurs in the TypeInferrer when a function returns multiple times. I have added something to go around the problem if the latter return type is the same as the first. See XXX comments in this commit: 613748f

Argument checking

I just got numba built/running on Windows for the first time, and tried the example in the readme. The resulting jitted code doesn't appear to check whether the arguments fit the required pattern:

In [3]: sum2d
Out[3]: <CFunctionType object at 0x0000000004C8F5F8>

In [4]: sum2d([1,2,3])
---------------------------------------------------------------------------
WindowsError                              Traceback (most recent call last)
<ipython-input-4-84532d040704> in <module>()
----> 1 sum2d([1,2,3])

WindowsError: exception: access violation reading 0x0000000000000003

In [5]: sum2d([1.0,2.0,3.0])
---------------------------------------------------------------------------
WindowsError                              Traceback (most recent call last)
<ipython-input-5-4e27170dcc75> in <module>()
----> 1 sum2d([1.0,2.0,3.0])

WindowsError: exception: access violation reading 0x0000000000000003

In [6]: import numpy

In [7]: sum2d(numpy.array([1.0,2.0,3.0]))
Out[7]: nan

In [8]: sum2d(numpy.array([[1.0,2.0,3.0]]))
Out[8]: 6.0

LLVM optimizer segfaults or module fails verification randomly if a jitted function has a docstring

The following test reproduces it:

import itertools
from llvm import core as lc, passes as lp
from numba import jit, int64, float64

def func_with_docstring(a,b,c,d):
    """some docstring"""
    return a+b+c+d


def test_jit_function_with_docstring():
    types = [int64, float64]
    for args in itertools.product(*([types]*4)):
        f = jit(argtypes=args, nopython=True)(func_with_docstring).lfunc
        optimize(f)


def optimize(f):
    pmb = lp.PassManagerBuilder.new()
    pmb.opt_level = 3
    pm = lp.PassManager.new()
    pm.add(f.module.owner.target_data)
    pmb.populate(pm)
    pm.run(f.module)


if __name__ == '__main__':
    test_jit_function_with_docstring()

It took me a while to isolate this since I'm further wrapping llvm functions generated by numba and the docstring was the last of my suspects. Anyway, if the docstring is removed from the function it jits fine.
If not optimization is preformed it works fine

The problem only occurs when registering several signatures.

Perhaps a llvm module constant is defined when generating code for the constant (in visit_Str) which llvm's optimizer removes from the module from seeing it unused but it is somehow referenced elsewhere? Just a wild guess...

Defining and returning an array within a jitted function seems to be broken

While trying Numba for some simple operations I came to this:

import numpy as np

from numba import d, int_
from numba.decorators import jit as jit

@jit(ret_type=d[:], arg_types=[d[:], int_])     
def calculate_moments(data, n_moments):

    rs = data.shape[0]
    result = np.zeros((n_moments, ))
    for r in range(rs):
        for m in range(n_moments):
            result[m] += data[r] ** m

    return result

data = np.random.random((1000,))
print(calculate_moments(data, 5))

output:

DEBUG -- translate:241:str_to_llvmtype
str_to_llvmtype(): str = ''
Traceback (most recent call last):
  File "m1.py", line 9, in <module>
    @jit(ret_type=d[:], arg_types=[d[:], int_])     
  File "/Users/grecco/bin/anaconda/lib/python2.7/site-packages/numba/decorators.py", line 212, in _jit
    t.translate()
  File "/Users/grecco/bin/anaconda/lib/python2.7/site-packages/numba/translate.py", line 951, in translate
    getattr(self, 'op_'+name)(i, op, arg)
  File "/Users/grecco/bin/anaconda/lib/python2.7/site-packages/numba/translate.py", line 1544, in op_CALL_FUNCTION
    func, args = self.func_resolve_type(func, args)
  File "/Users/grecco/bin/anaconda/lib/python2.7/site-packages/numba/translate.py", line 1234, in func_resolve_type
    lfunc = map_to_function(func.val, typs, self.mod)
  File "/Users/grecco/bin/anaconda/lib/python2.7/site-packages/numba/translate.py", line 139, in map_to_function
    typs = [str_to_llvmtype(x) if isinstance(x, str) else x for x in typs]
  File "/Users/grecco/bin/anaconda/lib/python2.7/site-packages/numba/translate.py", line 246, in str_to_llvmtype
    first_char = str[0]
IndexError: string index out of range

I think is related to have result as a np array. It is fine when I run the following (different purpose) code:

import numpy as np

from numba import d, int_
from numba.decorators import jit as jit

@jit(ret_type=d, arg_types=[d[:], int_])     
def calculate_moment(data, moment):

    rs = data.shape[0]
    result = 0.0
    for r in range(rs):
        result += data[r] ** moment

    return result

data = np.random.random((1000,))
print(calculate_moment(data, 5))

Mac OS X 10.6.8
Python 2.7.3 AnacondaCE (default, Sep 4 2012, 10:42:42)
numba sha1: 948c5b9

Ps.- After writing this, I realized that in the fbcorr example you give the output array

Integer type breaks compilation of function

Here's a straightforward implementation of dot. It's pretty slow, by the way :)

But if you change s = 0.0 to s = 0 below, things break immediately because of casting.

Since the types of A and B are known, I think

s = 0
s = s + A[i, k] + B[k, j]

should be able to figure out that s is a double, just like A and B.

Here's the dot, then:


import numpy as np

from numba.decorators import jit
from numba import float64, int32

@jit(arg_types=[float64[:, :], float64[:, :], float64[:, :]])
def ndot(A, B, out):
    rows_A, cols_A = A.shape
    rows_B, cols_B = B.shape

    # Take each row in A
    for i in range(rows_A):

        # And multiply by every column in B
        for j in range(cols_B):
            s = 0.0
            for k in range(cols_A):
                s = s + A[i, k] * B[k, j]

            out[i, j] = s

Mailing list/bug submission?

Hi,

thanks for writing this new tool for the python community. I was trying to run some benchmarks comparing Cython/Numpy/Shedskin/numexpr and wanted to see how Numba held up.

I tried following the examples as close as I could, but the code wouldn't run. Is there somewhere to discuss the bugs or a dev mailing list that is open to the public?

Federico

Problems with casting

I'm having problems with casting in numba. For example, the next code:

def avg2d(arr, result):
    M, N = arr.shape
    for i in range(M):
        avg = 0.
        for j in range(N):
            avg += arr[i,j]
        result[i] = avg / float(N)

gives me the next error:

Traceback (most recent call last):
File "examples/average.py", line 15, in <module>
  cavg2d = jit(arg_types=[d[:,:], d[:]])(avg2d)
File "/Users/faltet/Envs/env1/lib/python2.7/site-packages/numba/decorators.py", line 110, in _jit
  t.translate()
File "/Users/faltet/Envs/env1/lib/python2.7/site-packages/numba/translate.py", line 936, in translate
  getattr(self, 'op_'+name)(i, op, arg)
File "/Users/faltet/Envs/env1/lib/python2.7/site-packages/numba/translate.py", line 1492, in op_CALL_FUNCTION
  func, args = self.func_resolve_type(func, args)
File "/Users/faltet/Envs/env1/lib/python2.7/site-packages/numba/translate.py", line 1193, in func_resolve_type
  lfunc = map_to_function(func.val, typs, self.mod)
File "/Users/faltet/Envs/env1/lib/python2.7/site-packages/numba/translate.py", line 130, in map_to_function
  INTR = getattr(lc, 'INTR_%s' % func.__name__.upper())
AttributeError: 'module' object has no attribute 'INTR_FLOAT'

whereas I replace the line:

      result[i] = avg / float(N)

by:

      result[i] = avg / N*1.

I get another different error:

Result from numpy is 0.000351852223089 in 1.97410583496 (msec)
NumPy speed up is 560.101207729
Segmentation fault: 11

(the error being harder in this case...)

LLVM IR should not be dumped into the terminal

Numba is dumping the LLVM intermediate code whenever the compiler is invoked. Like this example:

def avg2d(arr, result):
    M, N = arr.shape
    for i in range(M):
        avg = 0.
        for j in range(N):
            avg += arr[i,j]
        result[i] = avg / N

cavg2d = jit(arg_types=[d[:,:], d[:]])(avg2d)

it outputs something like:

define double @avg2d({ i64, i32*, i8*, i32, i64*, i64*, i8*, i8*, i32, i8*, i8*, i8*, i64* }* %arr, { i64, i32*, i8*, i32, i64*, i64*, i8*, i8*, i32, i8*, i8*, i8*, i64* }* %result) {
Entry:
  %0 = getelementptr { i64, i32*, i8*, i32, i64*, i64*, i8*, i8*, i32, i8*, i8*, i8*, i64* }* %arr, i32 0, i32 4
  %1 = load i64** %0
  %2 = getelementptr i64* %1, i32 0
  %3 = load i64* %2
  %4 = getelementptr i64* %1, i32 1
  %5 = load i64* %4
  br label %BLOCK_18

This should not happen by default.

Complex numbers on windows

Running the tests suggests that things work well except for complex number support. Here are the results of test_all.py:

======================================================================
FAIL: test_get_complex_constant_fn (test_complex.TestComplex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Develop\numba\tests\test_complex.py", line 79, in test_get_complex_constant_fn
    compiled_get_complex_constant_fn())
AssertionError: (3-4j) != (2.5817550519390674e-316+1.8747554130430937e-316j)

======================================================================
FAIL: test_get_conj_fn (test_complex.TestComplex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Develop\numba\tests\test_complex.py", line 68, in test_get_conj_fn
    self.assertEqual(compiled_get_conj_fn(num0), 4 + 1.5j)
AssertionError: (2.5963042970778004e-316+2.5817546566865507e-316j) != (4+1.5j)

======================================================================
FAIL: test_get_imag_fn (test_complex.TestComplex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Develop\numba\tests\test_complex.py", line 56, in test_get_imag_fn
    self.assertEqual(compiled_get_imag_fn(num0), -2.)
AssertionError: 2.1219957905e-314 != -2.0

======================================================================
FAIL: test_get_real_fn (test_complex.TestComplex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Develop\numba\tests\test_complex.py", line 47, in test_get_real_fn
    self.assertEqual(compiled_get_real_fn(num0), 3.)
AssertionError: 2.539982e-316 != 3.0

======================================================================
FAIL: test_prod_sum_fn (test_complex.TestComplex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Develop\numba\tests\test_complex.py", line 93, in test_prod_sum_fn
    compiled_prod_sum_fn(a, x, b))
AssertionError: (-1+1j) != (1.8748265584960949e-316+2.4867138175374285e-315j)

----------------------------------------------------------------------
Ran 39 tests in 0.996s

FAILED (failures=5, skipped=1)

ControlFlowAnalysis chokes with a function with all returns inside if-esle btanches

The following test reproduces the probem with the latest HEAD (7759a7e)

from numba import jit

def test_can_jit_without_return_on_last_block():
    f1 = jit('d(i,i,i,i)')(with_)
    f2 = jit('d(i,i,i,i)')(without_)
    # smoke test
    for i in xrange(10):
        args = (i,)*4
        assert f1(*args)==f2(*args)

def without_(fmc, cbh, cbd, hhg):
    """
    Cannot be jitted, breaks with:

    Traceback (most recent call last):
      File "test.py", line 42, in <module>
        v()
      File "test.py", line 30, in test_can_jit_without_return_on_last_block
        f1 = jit('d(i,i,i,i)')(with_)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/decorators.py", line 269, in _jit2_decorator
        **kwargs)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/functions.py", line 186, in compile_function
        ctypes=ctypes, **kwds)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/pipeline.py", line 287, in compile
        context, func, restype, argtypes, codegen=True, **kwds)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/pipeline.py", line 256, in _infer_types
        return run_pipeline(context, func, ast, func_signature, **kwargs)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/pipeline.py", line 250, in run_pipeline
        return pipeline, pipeline.run_pipeline()
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/pipeline.py", line 136, in run_pipeline
        ast = getattr(self, method_name)(ast)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/pipeline.py", line 155, in cfg
        ast = transform.visit(ast)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/control_flow.py", line 1092, in visit
        return super(ControlFlowAnalysis, self).visit(node)
      File "/usr/lib/python2.7/ast.py", line 241, in visit
        return visitor(node)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/control_flow.py", line 1150, in visit_FunctionDef
        self.flow.compute_dominance_frontier()
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/control_flow.py", line 453, in compute_dominance_frontier
        block.idom = self.immediate_dominator(block)
      File "/home/alberto/Env/local/lib/python2.7/site-packages/numba/control_flow.py", line 436, in immediate_dominator
        assert len([b for b in candidates if len(b.dominators) == ndoms]) == 1
    AssertionError
    """
    icrit = (0.010*cbh*(460+25.9*fmc))**1.5
    if cbd<0.05:
        return -999
    elif icrit<0.01 or hhg < (icrit*0.5):
        return 0
    elif hhg < icrit:
        return 75 *((hhg-icrit*0.5)/(icrit*0.5))
    elif hhg < icrit*1.3:
        return 75 + 25*((hhg-icrit)/icrit)
    else:
        return 100

def with_(fmc, cbh, cbd, hhg):
    """If the last return is taken out of the last else block it can"""
    icrit = (0.010*cbh*(460+25.9*fmc))**1.5
    if cbd<0.05:
        return -999
    elif icrit<0.01 or hhg < (icrit*0.5):
        return 0
    elif hhg < icrit:
        return 75 *((hhg-icrit*0.5)/(icrit*0.5))
    elif hhg < icrit*1.3:
        return 75 + 25*((hhg-icrit)/icrit)
    return 100


if __name__ == '__main__':
    for k,v in vars().items():
        if k.startswith('test') and callable(v):
            print k
            v()

Meta not listed as a dependency

I was trying to run the numba test case but I got the following error:

ImportError: No module named meta.decompiler

I installed Meta and the testcase finally run without complainings. Meta should be stated as a dependency of numba.

Unary Operator Error - Negative (-) Symbol on Variables

Getting Unary Operator error when running this code:

def minimumFilter(image, filt):
    rows, columns = image.shape
    result = numpy.zeros(image.shape)

    filterSize = 5
    halfFilterSize = filterSize // 2    

    for x in range(halfFilterSize, rows - halfFilterSize):
        for y in range(halfFilterSize, columns - halfFilterSize):
            sMin = 255
            for j in range(-halfFilterSize, halfFilterSize):
                for i in range(-halfFilterSize, halfFilterSize):
                    filt[i + halfFilterSize, j + halfFilterSize] = image[x + i, y + j]
            for j in range(filterSize - 1):
                for i in range(filterSize - 1):
                    if filt[i, j] < sMin:
                         sMin = filt[i, j]
            result[x, y] = sMin

     return result

The problem is when I use:

-halfFilterSize

To fix the issue, I used:

(-1) * halfFilterSize

which executed fine. Just thought this should be fixed. It works fine with constant numbers, but not for variables.

Here is error output:

Traceback (most recent call last):
File "minimumFilter.py", line 9, in
@jit(argtypes=[int32[:,:], int32[:,:]], restype=int32[:,:])
File "/opt/anaconda/lib/python2.7/site-packages/numba/decorators.py", line 268, in _jit2_decorator
*_kwargs)
File "/opt/anaconda/lib/python2.7/site-packages/numba/functions.py", line 183, in compile_function
ctypes=ctypes, *_kwds)
File "/opt/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 169, in compile
context, func, restype, argtypes, codegen=True, *_kwds)
File "/opt/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 130, in _infer_types
return run_pipeline(context, func, ast, func_signature, *_kwargs)
File "/opt/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 124, in run_pipeline
return pipeline, pipeline.run_pipeline()
File "/opt/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 70, in run_pipeline
ast = getattr(self, method_name)(ast)
File "/opt/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 81, in type_infer
type_inferer.infer_types()
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 386, in infer_types
self.ast = self.visit(self.ast)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 481, in visit
result = super(TypeInferer, self).visit(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 297, in generic_visit
value = self.visit(value)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 481, in visit
result = super(TypeInferer, self).visit(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 597, in visit_For
self.visitlist(node.body)
File "/opt/anaconda/lib/python2.7/site-packages/numba/visitors.py", line 49, in visitlist
result = self.visit(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 481, in visit
result = super(TypeInferer, self).visit(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 597, in visit_For
self.visitlist(node.body)
File "/opt/anaconda/lib/python2.7/site-packages/numba/visitors.py", line 49, in visitlist
result = self.visit(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 481, in visit
result = super(TypeInferer, self).visit(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 588, in visit_For
node.iter = self.visit(node.iter)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 481, in visit
result = super(TypeInferer, self).visit(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 1001, in visit_Call
self.visitlist(node.args)
File "/opt/anaconda/lib/python2.7/site-packages/numba/visitors.py", line 49, in visitlist
result = self.visit(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 481, in visit
result = super(TypeInferer, self).visit(node)
File "/opt/anaconda/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/opt/anaconda/lib/python2.7/site-packages/numba/ast_type_inference.py", line 701, in visit_UnaryOp
node.variable = Variable(node.variable.type)
AttributeError: 'UnaryOp' object has no attribute 'variable'

Cython.Distutils not listed as a build requirement

Just tried building and installing the whole numba stack on a bare-bones dev machine with very few 3rd-party python packages following the instructions on the websites. When I tried building and installing numba itself, it fails with error:

$ python setup.py install
Traceback (most recent call last):
  File "setup.py", line 9, in <module>
    from Cython.Distutils import build_ext
ImportError: No module named Cython.Distutils

Not a major issue, it just needs to be listed as a build requirement in the instructions.

autojit decorator

Need an autojit decorator that is similar to @function but uses the bytecode translator pathway. The autojit decorator should monitor inputs and create functions on the fly if not already cached.

if u == U[i+j] creates error while U[i+j] == u works.

from numba import jit

@jit('i4(i4,f8,i4,f8[:])')
def FindMult(i,u,p,U):
    s = 0
    for j in range(0-p, p+2):
        if U[i+j] == u:
            s = s + 1
    return s

The above code works, but switching to u == U[i+j] raises an error:

Traceback (most recent call last):
File "findmulti.py", line 3, in
@jit('i4(i4,f8,i4,f8[:])')
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/decorators.py", line 268, in _jit2_decorator
*_kwargs)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/functions.py", line 185, in compile_function
ctypes=ctypes, *_kwds)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 209, in compile
context, func, restype, argtypes, codegen=True, *_kwds)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 170, in _infer_types
return run_pipeline(context, func, ast, func_signature, *_kwargs)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 164, in run_pipeline
return pipeline, pipeline.run_pipeline()
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 83, in run_pipeline
ast = getattr(self, method_name)(ast)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/pipeline.py", line 135, in codegen
self.translator.translate()
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 464, in translate
self.visit(node)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 359, in visit
return fn(node)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 745, in visit_For
self.generate_for_range(node, node.target, node.iter, node.body)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 961, in generate_for_range
self.visit(stmt)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 359, in visit
return fn(node)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 841, in visit_If
test = self.visit(node.test)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 359, in visit
return fn(node)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 812, in visit_Compare
lhs_lvalue, rhs_lvalue = self.visitlist([lhs, rhs])
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/visitors.py", line 79, in visitlist
return [self.visit(item) for item in list]
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 359, in visit
return fn(node)
File "/Users/travis/anaconda/lib/python2.7/site-packages/numba/ast_translate.py", line 1126, in visit_Subscript
raise error.InternalError(node, "Unsupported type:", node.value.type)
numba.error.InternalError: 7:16: Unsupported type: double[:]

Mutating input array causes core dump

Today was my first exposure to Numba, so please forgive my ignorance in advance.

I am trying to mutate the values of an array and return it like this:

import numpy as np

from numba.decorators import jit
from numba import int32


@jit(arg_types=[int32[:, :]], ret_type=int32[:, :])
def mutate_array(arr):
    m, n = arr.shape

    for i in range(m):
        for j in range(n):
            arr[i, j] = i + j

    return arr

arr = np.ones((7, 7), dtype='int32')

print mutate_array(arr)

This prints the correct value:

[[ 0  1  2  3  4  5  6]
 [ 1  2  3  4  5  6  7]
 [ 2  3  4  5  6  7  8]
 [ 3  4  5  6  7  8  9]
 [ 4  5  6  7  8  9 10]
 [ 5  6  7  8  9 10 11]
 [ 6  7  8  9 10 11 12]]

and then core dumps. Am I trying to do something that is unsupported?

ImportError: cannot import name _ext

In [1]: import numba
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-6d1098de51de> in <module>()
----> 1 import numba

/home/whyking/tools/numba/numba/__init__.py in <module>()
      8     raise
      9 
---> 10 from . import _numba_types
     11 from ._numba_types import *
     12 

/home/whyking/tools/numba/numba/_numba_types.py in <module>()
      7 # from numpy.ctypeslib import _typecodes
      8 
----> 9 from numba import llvm_types, _ext
     10 from numba.minivect.minitypes import *
     11 from numba.minivect import miniast, minitypes

ImportError: cannot import name _ext

`or` in if construct is not implemented

The following code will raise an error:

from numba import *
from numba.decorators import jit

@jit(restype=int32, argtypes=[int32])
def ifor(a):
    if a > 10 or a < -10:
        return 0
    else:
        return 1

traceback:

Traceback (most recent call last):
  File "ifor.py", line 5, in <module>
    @jit(restype=int32, argtypes=[int32])
  File "/home/michael/.local/lib/python2.7/site-packages/numba/decorators.py", line 207, in _jit
    t.translate()
  File "/home/michael/.local/lib/python2.7/site-packages/numba/translate.py", line 951, in translate
    getattr(self, 'op_'+name)(i, op, arg)
AttributeError: 'Translate' object has no attribute 'op_POP_JUMP_IF_TRUE'

Removing the or ... will work again

Issues using from within iPython

I can run the tests, I can also run the same code from the command line but trying to copy and paste the code directly into iPython fails as follows:

Python 2.7.2 (default, Jun 20 2012, 16:23:33)
Type "copyright", "credits" or "license" for more information.

IPython 0.14.dev -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import numpy as np

In [2]: from numba import double

In [3]: from numba.decorators import jit

In [4]:

In [4]: @jit(arg_types=[double[:,:]], ret_type = double)
   ...: def sum2d(arr):
   ...:         M, N = arr.shape
   ...:         result = 0.0
   ...:         for i in range(M):
   ...:                 for j in range(N):
   ...:                         result += arr[i,j]
   ...:                 return result
   ...:
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-4-e1f9177b3ba9> in <module>()
----> 1 @jit(arg_types=[double[:,:]], ret_type = double)
      2 def sum2d(arr):
      3         M, N = arr.shape
      4         result = 0.0
      5         for i in range(M):

/Library/Python/2.7/site-packages/numba/decorators.pyc in _jit(func)
    108                            "garbage collected!" % (func,))
    109         t = bytecode_translate.Translate(func, *args, **kws)
--> 110         t.translate()
    111         __tr_map__[func] = t
    112         return t.get_ctypes_func(llvm)

/Library/Python/2.7/site-packages/numba/translate.pyc in translate(self)
    940                     self.build_phi_nodes(self.crnt_block)
    941             if not is_dead_code:
--> 942                 getattr(self, 'op_'+name)(i, op, arg)
    943
    944         # Perform code optimization

/Library/Python/2.7/site-packages/numba/translate.pyc in op_STORE_FAST(self, i, op, arg)
   1325         newval = self.stack.pop(-1)
   1326         if isinstance(newval.val, DelayedObj):
-> 1327             self._generate_for_loop(i, op, arg, newval)
   1328         else:
   1329             if self.has_pending_phi(i, arg):

/Library/Python/2.7/site-packages/numba/translate.pyc in _generate_for_loop(self, i, op, arg, delayer)
   1252         xrange, or arange).'''
   1253         arg_val = self._locals[arg]
-> 1254         false_jump_target = self.pending_blocks.pop(i - 3)
   1255         crnt_block_data = self._revisit_block(i - 3)
   1256         inc_variable = delayer.val.get_inc()

KeyError: 34

System is OSX 10.8, running the repo versions of llvmpy, numba, and Meta.

llvm-py error: ‘printf’ was not declared in this scope

I'm trying to get numba working on my Gentoo Linux machine but I'm getting the following error:

% python setup.py build                                                                               (master⚡)
Using llvm-config=llvm-config
running build
running build_py
copying llvm/__init__.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/core.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/_intrinsic_ids.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/ee.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/passes.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/__init__.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/_util.py -> build/lib.linux-x86_64-2.7/llvm
copying llvm/core.py -> build/lib.linux-x86_64-2.7/llvm
running build_ext
building 'llvm._core' extension
x86_64-pc-linux-gnu-gcc -pthread -fPIC -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -D_GNU_SOURCE -I/usr/include -I/usr/include -I/usr/include/python2.7 -c llvm/_core.c -o build/temp.linux-x86_64-2.7/llvm/_core.o
x86_64-pc-linux-gnu-gcc -pthread -fPIC -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -D_GNU_SOURCE -I/usr/include -I/usr/include -I/usr/include/python2.7 -c llvm/wrap.c -o build/temp.linux-x86_64-2.7/llvm/wrap.o
x86_64-pc-linux-gnu-g++ -pthread -fPIC -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -D_GNU_SOURCE -I/usr/include -I/usr/include -I/usr/include/python2.7 -c llvm/extra.cpp -o build/temp.linux-x86_64-2.7/llvm/extra.o
llvm/extra.cpp: In function ‘unsigned int LLVMLoadLibraryPermanently(const char*, char**)’:
llvm/extra.cpp:433:55: error: ‘printf’ was not declared in this scope
error: command 'x86_64-pc-linux-gnu-g++' failed with exit status 1

with

% llvm-config --version                                                                               (master⚡)
3.1svn

Is this error familar to you ?

Confusion about calls into nonexistent math functions.

If I call the nonexistent math.abs() function, the translator gives a very confusing exception. It appears to assume the function is or will be there (since it does getattr(mod, 'abs')), and has a return type of PyObject *. If left alone, it gives the proper exception at call time (like it would in Python):

>>> from numba import *
>>> import math
>>> def bad_abs (x): return math.abs(x)
... 
>>> jitted_ba = autojit(bad_abs)
>>> jitted_ba(3.)
[...elided...]
Traceback (most recent call last):
[...elided...]
  File "/home/jriehl/git/numba/numba/decorators.py", line 167, in invoke_compiled
    return compiled_numba_func(*args, **kwargs)
AttributeError: 'module' object has no attribute 'abs'

If used in an arithmetic expression, it causes a translation error:

>>> from numba import *
>>> import math
>>> def bad_abs (x): return 4.5 * math.abs(x)
... 
>>> compiled_bad_abs = autojit(bad_abs)
>>> compiled_bad_abs(3.)
[...elided...]
Traceback (most recent call last):
[...elided...]
  File "/home/jriehl/git/numba/numba/ast_translate.py", line 1035, in visit_BinOp
    raise error.NumbaError(node, op, node.type, lhs, rhs)
numba.error.NumbaError: 1:0: <class '_ast.Mult'> PyObject *   %PyFloat_FromDouble_temp_load = load { i64, i32* }** %PyFloat_FromDouble_temp   %_temp_load = load { i64, i32* }** %_temp

So I guess this issue is either a bug against the bad type checking (I can see an argument for being conservative and emitting the getattr() instead of throwing a translation-time AttributeError), bad error reporting (this doesn't give much usable information from a user point of view), or an enhancement request for supporting native arithmetic operators.

Improved built-in math support.

Numba should know that when using functions like numpy.exp(), etc. they will return a scalar if passed a scalar. As such, the type system should not track their result as being a Python object.

(Note: this could be supported by nopython code generation, and which I have not checked.)

Complex example: ValueError: %s is not a simple value.

$ uname -a
Linux andy-imac-linuxdev 3.2.0-31-generic #50-Ubuntu SMP Fri Sep 7 16:16:45 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

In [10]: llvm.__version__ 
Out[10]: '0.8.4'

$ cd numba
$ git log -n 1
commit 9101f8a4f0d8ff77d46b70fbdf05b96ad30094a1
Author: Mark Florisson <[email protected]>
Date:   Wed Nov 7 15:56:49 2012 +0000

    Support indexing of C array types

$ ipython
Python 2.7.3 (default, Aug  1 2012, 05:14:39) 
Type "copyright", "credits" or "license" for more information.

IPython 0.14.dev -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from numba import autojit

In [2]: @autojit
   ...: def complex_support(real, imag):
   ...:     c = complex(real, imag)
   ...:     return (c**2).conjugate()
   ...: 

In [3]: c = 2.0 + 4.0j

In [4]: complex_support(c.real, c.imag), (c**2).conjugate()
value (2+4j)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-e6b15ac07fb0> in <module>()
----> 1 complex_support(c.real, c.imag), (c**2).conjugate()

/home/aterrel/opt/lib/python2.7/site-packages/numba/decorators.pyc in __call__(self, *args, **kwargs)
    162                 raise error.NumbaError("Expected %d arguments, got %d" % (
    163                                                         len(args), nargs))
--> 164             return self.wrapper(self, *args, **kwargs)
    165 
    166     def invoke_compiled(self, compiled_numba_func, *args, **kwargs):

/home/aterrel/opt/lib/python2.7/site-packages/numba/decorators.pyc in wrapper(numba_func, *args, **kwargs)
    190             dec = jit2(argtypes=types, target=target, nopython=nopython,
    191                        **translator_kwargs)
--> 192             compiled_numba_func = dec(f)
    193             return numba_func.invoke_compiled(compiled_numba_func, *args, **kwargs)
    194 

/home/aterrel/opt/lib/python2.7/site-packages/numba/decorators.pyc in _jit2_decorator(func)
    266                                                  llvm_module=_llvm_module,
    267                                                  llvm_ee=_llvm_ee,
--> 268                                                  **kwargs)
    269         signature, lfunc, wrapper_func = result
    270         return NumbaFunction(func, ctypes_func=wrapper_func,

/home/aterrel/opt/lib/python2.7/site-packages/numba/functions.pyc in compile_function(self, func, argtypes, restype, ctypes, **kwds)
    183                 func_signature, translator, ctypes_func = pipeline.compile(
    184                                 self.context, func, restype, argtypes,
--> 185                                 ctypes=ctypes, **kwds)
    186                 self.compiled_functions[func, tuple(func_signature.args)] = (
    187                                       func_signature, translator, ctypes_func)

/home/aterrel/opt/lib/python2.7/site-packages/numba/pipeline.pyc in compile(context, func, restype, argtypes, ctypes, compile_only, **kwds)
    237     """
    238     pipeline, (func_signature, symtab, ast) = _infer_types(
--> 239                 context, func, restype, argtypes, codegen=True, **kwds)
    240     t = pipeline.translator
    241 

/home/aterrel/opt/lib/python2.7/site-packages/numba/pipeline.pyc in _infer_types(context, func, restype, argtypes, **kwargs)
    206     func_signature = minitypes.FunctionType(return_type=restype,
    207                                             args=argtypes)
--> 208     return run_pipeline(context, func, ast, func_signature, **kwargs)
    209 
    210 def infer_types(context, func, restype=None, argtypes=None, **kwargs):

/home/aterrel/opt/lib/python2.7/site-packages/numba/pipeline.pyc in run_pipeline(context, func, ast, func_signature, pipeline, **kwargs)
    200     pipeline = pipeline or context.numba_pipeline(context, func, ast,
    201                                                   func_signature, **kwargs)
--> 202     return pipeline, pipeline.run_pipeline()
    203 
    204 def _infer_types(context, func, restype=None, argtypes=None, **kwargs):

/home/aterrel/opt/lib/python2.7/site-packages/numba/pipeline.pyc in run_pipeline(self)
    112 
    113             self._current_pipeline_stage = method_name
--> 114             ast = getattr(self, method_name)(ast)
    115 
    116         return self.func_signature, self.symtab, ast

/home/aterrel/opt/lib/python2.7/site-packages/numba/pipeline.pyc in const_folding(self, ast)
    127         const_folder = self.make_specializer(constant_folding.ConstantFolder,
    128                                              ast, constvars=constvars)
--> 129         return const_folder.visit(ast)
    130 
    131     def type_infer(self, ast):

/usr/lib/python2.7/ast.pyc in visit(self, node)
    239         method = 'visit_' + node.__class__.__name__
    240         visitor = getattr(self, method, self.generic_visit)
--> 241         return visitor(node)
    242 
    243     def generic_visit(self, node):

/usr/lib/python2.7/ast.pyc in generic_visit(self, node)
    295                 for value in old_value:
    296                     if isinstance(value, AST):
--> 297                         value = self.visit(value)
    298                         if value is None:
    299                             continue

/usr/lib/python2.7/ast.pyc in visit(self, node)
    239         method = 'visit_' + node.__class__.__name__
    240         visitor = getattr(self, method, self.generic_visit)
--> 241         return visitor(node)
    242 
    243     def generic_visit(self, node):

/usr/lib/python2.7/ast.pyc in generic_visit(self, node)
    304                 old_value[:] = new_values
    305             elif isinstance(old_value, AST):
--> 306                 new_node = self.visit(old_value)
    307                 if new_node is None:
    308                     delattr(node, field)

/usr/lib/python2.7/ast.pyc in visit(self, node)
    239         method = 'visit_' + node.__class__.__name__
    240         visitor = getattr(self, method, self.generic_visit)
--> 241         return visitor(node)
    242 
    243     def generic_visit(self, node):

/usr/lib/python2.7/ast.pyc in generic_visit(self, node)
    304                 old_value[:] = new_values
    305             elif isinstance(old_value, AST):
--> 306                 new_node = self.visit(old_value)
    307                 if new_node is None:
    308                     delattr(node, field)

/usr/lib/python2.7/ast.pyc in visit(self, node)
    239         method = 'visit_' + node.__class__.__name__
    240         visitor = getattr(self, method, self.generic_visit)
--> 241         return visitor(node)
    242 
    243     def generic_visit(self, node):

/usr/lib/python2.7/ast.pyc in generic_visit(self, node)
    304                 old_value[:] = new_values
    305             elif isinstance(old_value, AST):
--> 306                 new_node = self.visit(old_value)
    307                 if new_node is None:
    308                     delattr(node, field)

/usr/lib/python2.7/ast.pyc in visit(self, node)
    239         method = 'visit_' + node.__class__.__name__
    240         visitor = getattr(self, method, self.generic_visit)
--> 241         return visitor(node)
    242 
    243     def generic_visit(self, node):

/home/aterrel/opt/lib/python2.7/site-packages/numba/ast_constant_folding.pyc in visit_BinOp(self, node)
    204 
    205         if self.is_constant(lval) and self.is_constant(rval):
--> 206             return self.eval_binary_operation(node.op, lval, rval)
    207         else:
    208             return node

/home/aterrel/opt/lib/python2.7/site-packages/numba/ast_constant_folding.pyc in eval_binary_operation(self, op, left, right)
    272         operator = ast_to_binary_operator[type(op)]
    273         func = compile_time_binary_operators[operator]
--> 274         ret = func(self.valueof(left), self.valueof(right))
    275         if ret is True:
    276             node = ast.Name(id='True', ctx=ast.Load())

/home/aterrel/opt/lib/python2.7/site-packages/numba/ast_constant_folding.pyc in valueof(self, node)
    301                 print 'value', value
    302                 if not is_simple_value(value):
--> 303                     raise ValueError("%s is not a simple value.")
    304                 return value
    305         raise ValueError("node %s is not a has constant value" % node)

ValueError: %s is not a simple value.

In [5]:

Inconsistent Values from Numba

I'm running a forward-euler diffusion solver with a small sink. I seem to be getting different values depending on the number of inner loops called within a jitted numba func.

code can be found here:
https://gist.github.com/3788085

output:
With u[Lx/2,Ly/2]= 1000 set iterNum to 10 and iterate 10 times
0 : Value in center: 83.2322272
1 : Value in center: 40.1092423469
2 : Value in center: 25.8394872234
3 : Value in center: 18.5599203731
4 : Value in center: 14.1658857468
5 : Value in center: nan
6 : Value in center: nan
7 : Value in center: nan
8 : Value in center: nan
9 : Value in center: nan
reset lattice, set iterNum to 100 and iterate once

With u[Lx/2,Ly/2]= 1000 set iterNum to 20 and iterate 5 times
0 : Value in center: 40.1092423469
1 : Value in center: 18.5599203731
2 : Value in center: 11.2576765715
3 : Value in center: 7.7119173751
4 : Value in center: 5.68337462895
reset lattice, set iterNum to 100 and iterate once

Value in center: 5.68337462895
reset lattice, run 100 iterations without numba

Value in center: 5.68337462895

Numba version: latest git pull

Config 1:
OS X 10.7.4
Macbook Air
Python 2.7.3

Config 2:
Debian 64 - 3.2.0-3
python 2.7.3
Lenovo X200

if statement inside a for loop fails to compile

Putting an if statement fails inside of a for loop.
I was writing some "find the index of the closest vector" code which wouldn't compile, and found this to be the source of the problem.

This works:

    from numba.decorators import jit
    import numba
    @jit(arg_types=[numba.double], ret_type=numba.double)
    def is_five(some_value):
        if some_value == 5.0:
            return 1.0
        else:
            return 0.0

This fails:

    @jit(arg_types=[numba.double], ret_type=numba.double)
    def is_REALLY_five(some_value):
        for i in range(5):
            if some_value == 5.0:
                return 1.0
        return 0.0

This is how it fails:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-3b23f7331ceb> in <module>()
----> 1 @jit(arg_types=[numba.double], ret_type=numba.double)
      2 def is_REALLY_five(some_value):
      3     for i in range(5):
      4         if some_value == 5.0:
      5             return 1.0

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/decorators.pyc in _jit(func)
     77                   "garbage collected!" % (func,))
     78         t = Translate(func, *args, **kws)
---> 79         t.translate()
     80         __tr_map__[func] = t
     81         return t.get_ctypes_func(llvm)

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/translate.pyc in translate(self)
    926                     # Ensure we are playing with locals that might
    927                     # actually precede the next block.
--> 928                     self.check_locals(i)
    929 
    930                 self.crnt_block = i

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/translate.pyc in check_locals(self, i)
   1135                 else:
   1136                     assert next_locals is not None, "Internal compiler error!"
-> 1137             self._locals = next_locals[:]
   1138 
   1139     def get_ctypes_func(self, llvm=True):

TypeError: 'NoneType' object has no attribute '__getitem__'

Python target version?

It would be nice to mention target python version (or at least 2.x vs 3.x) somewhere in then README. Thank you!

Variable.ltype does not respect user-defined context

Variable.ltype should not use utils.context

@property
def ltype(self):
    """
    The LLVM type for the type of this variable or LLVM Value.
    """
    if self.lvalue is not None:
        return self.lvalue.type
    return self.type.to_llvm(utils.context)

examples/sum.py fails on OSX 10.7

Running 10.7. Python 2.7.3 installed from 64-bit EPD 7.3.
All other examples work.

➜ examples git:(master) python sum.py
CONVERTING double
CONVERTING double[:, :]
CONVERTING double
['d']
str_to_llvmtype(): str = 'f64'
{'blocks': {0: <llvm.core.BasicBlock object at 0x1021c1650>,
24: <llvm.core.BasicBlock object at 0x1021c16d0>,
34: <llvm.core.BasicBlock object at 0x1021c1750>,
37: <llvm.core.BasicBlock object at 0x1021c17d0>,
40: <llvm.core.BasicBlock object at 0x1021c18d0>,
43: <llvm.core.BasicBlock object at 0x1021c1950>,
53: <llvm.core.BasicBlock object at 0x1021c19d0>,
56: <llvm.core.BasicBlock object at 0x1021c1a50>,
59: <llvm.core.BasicBlock object at 0x1021c1b50>,
82: <llvm.core.BasicBlock object at 0x1021c1ad0>,
86: <llvm.core.BasicBlock object at 0x1021c1850>},
'blocks_dom': {0: set([0]),
24: set([0, 24]),
34: set([0, 24, 34, 37, 40, 43, 56, 82]),
37: set([0, 24, 37]),
40: set([0, 24, 37, 40]),
43: set([0, 24, 37, 40, 43]),
53: set([0, 24, 37, 40, 43, 53, 56, 59]),
56: set([0, 24, 37, 40, 43, 56]),
59: set([0, 24, 37, 40, 43, 56, 59]),
82: set([0, 24, 37, 40, 43, 56, 82]),
86: set([0, 24, 37, 86])},
'blocks_in': {0: set(),
24: set([0]),
34: set([82]),
37: set([24, 34]),
40: set([37]),
43: set([40]),
53: set([59]),
56: set([43, 53]),
59: set([56]),
82: set([56]),
86: set([37])},
'blocks_out': {0: set([24]),
24: set([37]),
34: set([37]),
37: set([40, 86]),
40: set([43]),
43: set([56]),
53: set([56]),
56: set([59, 82]),
59: set([53]),
82: set([34]),
86: set()},
'blocks_reaching': {0: set([0]),
24: set([0, 24]),
34: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
37: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
40: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
43: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
53: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
56: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
59: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
82: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82]),
86: set([0, 24, 34, 37, 40, 43, 53, 56, 59, 82, 86])},
'blocks_reads': {0: set([0]),
24: set([1]),
34: set(),
37: set([4]),
40: set(),
43: set([2]),
53: set(),
56: set([5]),
59: set([0, 3, 4, 5]),
82: set(),
86: set([3])},
'blocks_writer': {0: {1: 9, 2: 12, 3: 18},
24: {4: 33},
34: {4: 34},
37: {3: 37, 4: 37},
40: {},
43: {5: 52},
53: {5: 53},
56: {3: 56, 5: 56},
59: {3: 76},
82: {},
86: {}},
'blocks_writes': {0: set([0, 1, 2, 3]),
24: set([4]),
34: set([4]),
37: set([3, 4]),
40: set(),
43: set([5]),
53: set([5]),
56: set([3, 5]),
59: set([3]),
82: set(),
86: set()},
'translator': <numba.translate.Translate object at 0x1021c1410>}
op_LOAD_ATTR(): 3 106 shape <Variable(val=<llvm.core.Argument object at 0x1021c15d0>, llvm=<llvm.core.Argument object at 0x1021c15d0>, typ='arr[f64]')> arr[f64]
op_LOAD_ATTR(): { i64, i32, i8_, i32, i64_, i64_, i8_, i8_, i32, i8_, i8_, i8_, i64_ }*
('op_CALL_FUNCTION():', <Variable(val=, _llvm=None, typ=['func'])>)
str_to_llvmtype(): str = 'f64'
add_phi_incomming(): reaching_defs = {24: {0: 0, 1: 0, 2: 0, 3: 0, 4: 24},
34: {0: 0, 1: 0, 2: 0, 3: 56, 4: 34, 5: 56}}
crnt_block=37, pred=34, local=3
str_to_llvmtype(): str = 'i64'
add_phi_incomming(): reaching_defs = {24: {0: 0, 1: 0, 2: 0, 3: 0, 4: 24},
34: {0: 0, 1: 0, 2: 0, 3: 56, 4: 34, 5: 56}}
crnt_block=37, pred=34, local=4
op_BINARY_ADD(): <Variable(val=<llvm.core.PHINode object at 0x1021c1fd0>, _llvm=<llvm.core.PHINode object at 0x1021c1fd0>, typ='i64')> + <Variable(val=<llvm.core.ConstantInt object at 0x1021c1f50>, _llvm=<llvm.core.ConstantInt object at 0x1021c1f50>, typ='i64')>
resolve_type(): arg1 = <Variable(val=<llvm.core.PHINode object at 0x1021c1fd0>, _llvm=<llvm.core.PHINode object at 0x1021c1fd0>, typ='i64')>, arg2 = <Variable(val=<llvm.core.ConstantInt object at 0x1021c1f50>, _llvm=<llvm.core.ConstantInt object at 0x1021c1f50>, typ='i64')>
resolve_type() ==> 'i64'
resolve_type(): arg1 = <Variable(val=<llvm.core.PHINode object at 0x1021c1fd0>, _llvm=<llvm.core.PHINode object at 0x1021c1fd0>, typ='i64')>, arg2 = <Variable(val=<llvm.core.Instruction object at 0x1021c1c10>, _llvm=<llvm.core.Instruction object at 0x1021c1c10>, typ='i64')>
resolve_type() ==> 'i64'
('op_CALL_FUNCTION():', <Variable(val=, _llvm=None, typ=['func'])>)
str_to_llvmtype(): str = 'f64'
add_phi_incomming(): reaching_defs = {43: {0: 0, 1: 0, 2: 0, 3: 37, 4: 37, 5: 43},
53: {0: 0, 1: 0, 2: 0, 3: 59, 4: 37, 5: 53}}
crnt_block=56, pred=53, local=3
str_to_llvmtype(): str = 'i64'
add_phi_incomming(): reaching_defs = {43: {0: 0, 1: 0, 2: 0, 3: 37, 4: 37, 5: 43},
53: {0: 0, 1: 0, 2: 0, 3: 59, 4: 37, 5: 53}}
crnt_block=56, pred=53, local=5
op_BINARY_ADD(): <Variable(val=<llvm.core.PHINode object at 0x1021d5290>, _llvm=<llvm.core.PHINode object at 0x1021d5290>, typ='i64')> + <Variable(val=<llvm.core.ConstantInt object at 0x1021d5250>, _llvm=<llvm.core.ConstantInt object at 0x1021d5250>, typ='i64')>
resolve_type(): arg1 = <Variable(val=<llvm.core.PHINode object at 0x1021d5290>, _llvm=<llvm.core.PHINode object at 0x1021d5290>, typ='i64')>, arg2 = <Variable(val=<llvm.core.ConstantInt object at 0x1021d5250>, _llvm=<llvm.core.ConstantInt object at 0x1021d5250>, typ='i64')>
resolve_type() ==> 'i64'
resolve_type(): arg1 = <Variable(val=<llvm.core.PHINode object at 0x1021d5290>, _llvm=<llvm.core.PHINode object at 0x1021d5290>, typ='i64')>, arg2 = <Variable(val=<llvm.core.Instruction object at 0x1021c1d90>, _llvm=<llvm.core.Instruction object at 0x1021c1d90>, typ='i64')>
resolve_type() ==> 'i64'
op_BINARY_SUBSCR(): arr_var.typ = arr[f64]
str_to_llvmtype(): str = 'f64'
%24 = load double* %23
op_BINARY_ADD(): <Variable(val=<llvm.core.PHINode object at 0x1021c1d10>, _llvm=<llvm.core.PHINode object at 0x1021c1d10>, typ='f64')> + <Variable(val=<llvm.core.Instruction object at 0x1021d5650>, _llvm=<llvm.core.Instruction object at 0x1021d5650>, typ='f64')>
resolve_type(): arg1 = <Variable(val=<llvm.core.PHINode object at 0x1021c1d10>, _llvm=<llvm.core.PHINode object at 0x1021c1d10>, typ='f64')>, arg2 = <Variable(val=<llvm.core.Instruction object at 0x1021d5650>, _llvm=<llvm.core.Instruction object at 0x1021d5650>, typ='f64')>
resolve_type() ==> 'f64'
; ModuleID = 'csum_mod_1021c1410'

define double @csum({ i64, i32_, i8_, i32, i64_, i64_, i8_, i8_, i32, i8_, i8_, i8_, i64_ }* %arr) {
Entry:
%0 = getelementptr { i64, i32_, i8_, i32, i64_, i64_, i8_, i8_, i32, i8_, i8_, i8_, i64_ }* %arr, i32 0, i32 4
%1 = load i64** %0
%2 = getelementptr i64* %1, i32 0
%3 = load i64* %2
%4 = getelementptr i64* %1, i32 1
%5 = load i64* %4
br label %BLOCK_24

BLOCK_24: ; preds = %Entry
br label %BLOCK_37

BLOCK_34: ; preds = %BLOCK_82
%6 = add i64 %8, 1
br label %BLOCK_37

BLOCK_37: ; preds = %BLOCK_34, %BLOCK_24
%7 = phi double [ 0.000000e+00, %BLOCK_24 ], [ %11, %BLOCK_34 ]
%8 = phi i64 [ 0, %BLOCK_24 ], [ %6, %BLOCK_34 ]
%9 = icmp slt i64 %8, %3
br i1 %9, label %BLOCK_40, label %BLOCK_86

BLOCK_86: ; preds = %BLOCK_37
ret double %7

BLOCK_40: ; preds = %BLOCK_37
br label %BLOCK_43

BLOCK_43: ; preds = %BLOCK_40
br label %BLOCK_56

BLOCK_53: ; preds = %BLOCK_59
%10 = add i64 %12, 1
br label %BLOCK_56

BLOCK_56: ; preds = %BLOCK_53, %BLOCK_43
%11 = phi double [ %7, %BLOCK_43 ], [ %25, %BLOCK_53 ]
%12 = phi i64 [ 0, %BLOCK_43 ], [ %10, %BLOCK_53 ]
%13 = icmp slt i64 %12, %5
br i1 %13, label %BLOCK_59, label %BLOCK_82

BLOCK_82: ; preds = %BLOCK_56
br label %BLOCK_34

BLOCK_59: ; preds = %BLOCK_56
%14 = getelementptr { i64, i32_, i8_, i32, i64_, i64_, i8_, i8_, i32, i8_, i8_, i8_, i64_ }* %arr, i32 0, i32 5
%15 = load i64** %14
%16 = getelementptr i64* %15, i32 0
%17 = load i64* %16
%18 = mul i64 %8, %17
%19 = getelementptr { i64, i32_, i8_, i32, i64_, i64_, i8_, i8_, i32, i8_, i8_, i8_, i64_ }* %arr, i32 0, i32 2
%20 = load i8** %19
%21 = getelementptr i8* %20, i64 %18
%22 = bitcast i8* %21 to double*
%23 = getelementptr double* %22, i64 %12
%24 = load double* %23
%25 = fadd double %11, %24
br label %BLOCK_53
}

Traceback (most recent call last):
File "sum.py", line 13, in
csum = jit(ret_type=d, arg_types=[d[:,:]])(sum)
File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/decorators.py", line 78, in _jit
t = Translate(func, _args, *_kws)
File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/translate.py", line 814, in init
self.fco = func.func_code
AttributeError: 'builtin_function_or_method' object has no attribute 'func_code'

Confusing error with wrong number of arguments

Based on reading the source, I think the error message on decorators.py(163) has the parameters in the wrong order (that is, it complains "NumbaError: Expected 2 arguments, got 3" when in fact two were given and the function takes three).

Abort error on 64-bit Linux running mandel.py

When running examples/mandel.py on 64-bit Linux.

python: /home/ilan/aroot/work/llvm-3.1.src/include/llvm/CodeGen/LiveInterval.h:130: llvm::LiveRange::LiveRange(llvm::SlotIndex, llvm::SlotIndex, llvm::VNInfo*): As
sertion `S < E && "Cannot create empty or backwards range"' failed.
Stack dump:
0. Running pass 'Greedy Register Allocator' on function '@__numba_wrapper___numba_specialized_create_fractal_double_double_double_double_uint8_5B__3A__2C___3A
__3A_1_5D__int'

This is on Wakari whose /etc/issue is:

CentOS release 6.3 (Final)
Kernel \r on an \m

The Python is 64-bit.

This is the same error Andy saw on Unix on TACC machines.

Can't multiply two arrays?

Hi numba developers. I only just learned about this project and it looks extremely interesting. I got everything to build fine and the examples work great. I then went to write my own test to see if there is any speed-up with multiplying two arrays in numba vs. in numpy.

However, I get an error that suggests that multiplication of doubles is not yet supported? I just wanted to make sure I'm interpreting this correctly.

I'm not sure if this is really an ``Issue", but I wasn't sure where else to post . Thanks!

--- snip ---
numba.error.NumbaError: 23:20: Binary operations mul on values typed double[:] and double[:] not (yet) supported
--- snip ---

Here's my code, inspired by the current examples.

--- snip ---

from numpy import random,zeros
from numba import double
from numba.decorators import jit as jit

import time

def mult2arr_np(x,y):

result = x*y

return result

def mult2arr(x,y):

ix = len(x)
jy = len(y)
result = zeros(ix)
for i in range(ix):
    result[i] = x[i]*y[i]

return result

cmult2arr = jit(restype=double[:,:], argtypes=[double[:,:],double[:,:]])(mult2arr)

x = random.random(100000)
y = random.random(100000)

start = time.time()
res = mult2arr(x,y)
duration = time.time() - start
print "Result from python is %s in %s (msec)" % (res, duration*1000)

start = time.time()
res = cmult2arr(x,y)
duration2 = time.time() - start
print "Result from compiled is %s in %s (msec)" % (res, duration2*1000)

print "Speed up is %s" % (duration / duration2)

start = time.time()
res = mult2arr_np(x,y)
duration3 = time.time() - start
print "Result from numpy is %s in %s (msec)" % (res, duration3*1000)

llvm.LLVMException: PHI node entries do not match predecessors!

From this shortish script calling the ext_force2 function gives an error

from numba import autojit

from numpy import zeros, double

nple = 64
k_fene = 15.0
R0 = 2.0
R0_2 = R0*R0

sigma = 1.0
sigma2 = sigma*sigma
sigma6 = sigma2*sigma2*sigma2
sigma12 = sigma6*sigma6

dpart = 1.12246*sigma
rcut = dpart
r2cut = dpart*dpart
k_fene = 15.0
force = 5.0 # Constant bias force in ext_force
width = 0.8775  # for ext force

@autojit()
def ext_force1(x, y, extx, exty):
    for i in xrange(nple):
        if abs(x[i]) > rcut:
            extx[i] = 0.0
        elif abs(y[i]) > rcut + width:
            r2dist = 1.0 / (x[i] * x[i])
            r6dist = r2dist * r2dist * r2dist
            extx[i] = (x[i] * 48.0 * r6dist * r2dist *
                       (sigma12 * r6dist - 0.5 * sigma6))
        else:
            ydista = y[i] - width - rcut
            ydistb = y[i] + width + rcut
            dist2a = x[i] * x[i] + ydista * ydista
            dist2b = x[i] * x[i] + ydistb * ydistb
            if dist2b < r2cut or dist2a < r2cut:
                if dist2b < r2cut:
                    ydist = ydistb
                    r2dist = 1.0 / dist2b
                else:
                    ydist = ydista
                    r2dist = 1.0 / dist2a
                r6dist = r2dist * r2dist * r2dist
                lj_factor = 48.0 * r6dist * r2dist * (sigma12 * r6dist - 0.5 * sigma6)
                extx[i] = x[i] * lj_factor
                exty[i] = ydist * lj_factor
        if abs(x[i]) < 0.5 and abs(y[i]) < width + rcut:
            extx[i] += force  # constant bias force in the x-direction
    return extx, exty


@autojit()
def ext_force2(x, y):
    extx = zeros(nple, double)
    exty = zeros(nple, double)
    for i in xrange(nple):
        if abs(x[i]) > rcut:
            extx[i] = 0.0
        elif abs(y[i]) > rcut + width:
            r2dist = 1.0 / (x[i] * x[i])
            r6dist = r2dist * r2dist * r2dist
            extx[i] = (x[i] * 48.0 * r6dist * r2dist *
                       (sigma12 * r6dist - 0.5 * sigma6))
        else:
            ydista = y[i] - width - rcut
            ydistb = y[i] + width + rcut
            dist2a = x[i] * x[i] + ydista * ydista
            dist2b = x[i] * x[i] + ydistb * ydistb
            if dist2b < r2cut or dist2a < r2cut:
                if dist2b < r2cut:
                    ydist = ydistb
                    r2dist = 1.0 / dist2b
                else:
                    ydist = ydista
                    r2dist = 1.0 / dist2a
                r6dist = r2dist * r2dist * r2dist
                lj_factor = 48.0 * r6dist * r2dist * (sigma12 * r6dist - 0.5 * sigma6)
                extx[i] = x[i] * lj_factor
                exty[i] = ydist * lj_factor
        if abs(x[i]) < 0.5 and abs(y[i]) < width + rcut:
            extx[i] += force  # constant bias force in the x-direction
    return extx, exty


from numpy.random import randn

extx = zeros(nple, double)
exty = zeros(nple, double)

print ext_force1(randn(nple), randn(nple), extx, exty)
print ext_force2(randn(nple), randn(nple))

Traceback (most recent call last):
  File "test2.py", line 92, in <module>
    print ext_force2(randn(nple), randn(nple))
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/decorators.py", line 164, in __call__
    return self.wrapper(self, *args, **kwargs)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/decorators.py", line 189, in wrapper
    compiled_numba_func = dec(f)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/decorators.py", line 272, in _jit2_decorator
    **kwargs)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/functions.py", line 186, in compile_function
    ctypes=ctypes, **kwds)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/pipeline.py", line 287, in compile
    context, func, restype, argtypes, codegen=True, **kwds)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/pipeline.py", line 256, in _infer_types
    return run_pipeline(context, func, ast, func_signature, **kwargs)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/pipeline.py", line 250, in run_pipeline
    return pipeline, pipeline.run_pipeline()
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/pipeline.py", line 136, in run_pipeline
    ast = getattr(self, method_name)(ast)
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/pipeline.py", line 223, in codegen
    self.translator.translate()
  File "/home/hukka/.local/lib/python2.7/site-packages/numba/ast_translate.py", line 630, in translate
    self.mod.verify()  # only Module level verification checks everything.
  File "/home/hukka/.local/lib/python2.7/site-packages/llvm/core.py", line 521, in verify
    raise llvm.LLVMException(ret)
llvm.LLVMException: PHI node entries do not match predecessors!
  %r2dist_6 = phi double [ %r2dist_5, %"exit_if_71:16" ], [ %r2dist_1, %or.done ]
label %"exit_if_71:16"
label %"no_error_54:059"
Broken module found, compilation terminated.
Broken module found, compilation terminated.

In the issue #39 the comment says that returning arrays does not work yet, but since the docs (http://numba.pydata.org/numba-doc/dev/doc/examples.html#id1) show that arrays can be created and returned from a jitted func, I'm guessing there is something else broken here?

Error with 1d-array/list access

from numba import jit
from numpy import zeros

@jit()
def test():
    foo = zeros((1, 1))
    foo[0, 0] = 0

test()

works fine, but

@jit()
def test():
    foo = zeros((1,))
    foo[0] = 0

@jit()
def test():
    foo = [0]
    foo[0] = 0

fails with NotImplementedError: ('i32', '{ i64, i32* }*').

Seems like all the examples use multidimensional arrays.

5 LLVM Modules were built during import. Are they ever used?

I placed a traceback at llvm.core.Module.new to track where modules are built. Five were built just when numba is imported. We need to verify if they are used or not; and, remove all the unnecessary module building.

python -c "from numba import *"

The traceback result:

Module default_module
  File "<string>", line 1, in <module>
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/__init__.py", line 51, in <module>
    from . import decorators
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 9, in <module>
    from . import utils, functions, ast_translate as translate, ast_type_inference
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/utils.py", line 63, in <module>
    context = get_minivect_context()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/utils.py", line 61, in get_minivect_context
    return NumbaContext()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/minivect/miniast.py", line 137, in __init__
    self.llvm_module = llvm.core.Module.new('default_module')
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/llvm/core.py", line 345, in new
    traceback.print_stack()


Module default_module
  File "<string>", line 1, in <module>
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/__init__.py", line 51, in <module>
    from . import decorators
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 9, in <module>
    from . import utils, functions, ast_translate as translate, ast_type_inference
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/functions.py", line 7, in <module>
    import numba.ast_translate as translate
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/ast_translate.py", line 16, in <module>
    from . import visitors, nodes, llvm_types, utils
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/nodes.py", line 17, in <module>
    context = utils.get_minivect_context()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/utils.py", line 61, in get_minivect_context
    return NumbaContext()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/minivect/miniast.py", line 137, in __init__
    self.llvm_module = llvm.core.Module.new('default_module')
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/llvm/core.py", line 345, in new
    traceback.print_stack()


Module default
  File "<string>", line 1, in <module>
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/__init__.py", line 51, in <module>
    from . import decorators
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 18, in <module>
    default_module = _lc.Module.new('default')
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/llvm/core.py", line 345, in new
    traceback.print_stack()


Module default_module
  File "<string>", line 1, in <module>
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/__init__.py", line 51, in <module>
    from . import decorators
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 110, in <module>
    context = utils.get_minivect_context()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/utils.py", line 61, in get_minivect_context
    return NumbaContext()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/minivect/miniast.py", line 137, in __init__
    self.llvm_module = llvm.core.Module.new('default_module')
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/llvm/core.py", line 345, in new
    traceback.print_stack()


Module default
  File "<string>", line 1, in <module>
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/__init__.py", line 51, in <module>
    from . import decorators
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/decorators.py", line 111, in <module>
    context.llvm_context = translate.LLVMContextManager()
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/ast_translate.py", line 142, in __init__
    self._initialize(opt=opt)
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/ast_translate.py", line 148, in _initialize
    default_mod = cls._init_module(cls._DEFAULT_MODULE, opt=opt)
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/numba/ast_translate.py", line 160, in _init_module
    mod = lc.Module.new(name)
  File "/Users/sklam/Library/Python/2.7/lib/python/site-packages/llvm/core.py", line 345, in new
    traceback.print_stack()

Support object arithmetic.

A user reported the following as not working:

@autojit
def maxstar1d(a, b):
    M = a.shape[0]
    res = numpy.empty(M)
    for i in range(M):
        res[i] = numpy.max(a[i], b[i]) + numpy.log1p(
            numpy.exp(-numpy.abs(a[i] - b[i])))
    return res

This is now a (skipped) test case in numba.tests.test_unary_ops (commit d3325c3).

The root cause is the built-in math functions are indicated as returning Python objects, and object arithmetic is not supported. Support for object arithmetic should be added and this test should pass.

Providing more optimal code generation should also be documented as another enhancement.

numba.i not being converted to the correct type string in the bytecode translator.

See:

https://groups.google.com/a/continuum.io/d/topic/numba-users/Fm8FPREdXHQ/discussion

The numba.translate.map_to_strtype() function is mapping the Minivect integer type (numba.i) to 'i32' (which is correct for the internal type representation for the bytecode interpreter), not 'i4' (which is correct for Numpy and numba.translate.convert_to_ctypes()).

numpy.zeros not recognized?

Hello,

The follwing bare bone function seems to fail to understand numpy.zeros.

I see in the example you could use zero_like maybe, but in my case my output is 1D greater than any input, so I need to create it via a numpy.zeros or similar.

I left the actual function I'm trying to implement commented out, if naybody can comment on obvious numba errors with it please feel free to do so.

Also if I type n1=n2=n3=n4=10 the jit doesn't understand.

import numba
from numba.decorators import jit as jit

def recontruct4DCore(ps,A,B,P0):
## n1,n3,n4 = ps.shape
## n2=A.shape[0]
n1=10
n2=10
n3=10
n4=10
p = numpy.zeros((n1,n2,n3,n4))
## for j in range(n2):
## Afact=A[j]_P0
## for i in range(n1):
## for k in range(n3):
## for l in range(n4):
## p[i,j,k,l]=ps[i,k,l]_B[j]+Afact
return p

numbaReconstruct4D = jit(ret_type = numba.d[:,:,:,:], arg_types=[numba.d[:,:,:],numba.d[:],numba.d[:],numba.d])(recontruct4DCore)

@vectorize'd functions not compiling, crashing kernel

In [1]: from numba.decorators import vectorize

In [2]: @vectorize
...: def my_add(x):
...: return x+x
...:
str_to_llvmtype(): str = 'f64'
str_to_llvmtype(): str = 'f64'
{'blocks': {0: <llvm.core.BasicBlock object at 0x10afbdbd0>},
'blocks_dom': {0: set([0])},
'blocks_in': {0: set()},
'blocks_out': {0: set()},
'blocks_reaching': {0: set([0])},
'blocks_reads': {0: set([0])},
'blocks_writer': {0: {}},
'blocks_writes': {0: set([0])},
'translator': <numba.translate.Translate object at 0x10afbd910>}
op_BINARY_ADD(): <Variable(val=<llvm.core.Argument object at 0x10afbdb50>, _llvm=<llvm.core.Argument object at 0x10afbdb50>, typ='f64')> + <Variable(val=<llvm.core.Argument object at 0x10afbdb50>, _llvm=<llvm.core.Argument object at 0x10afbdb50>, typ='f64')>
resolve_type(): arg1 = <Variable(val=<llvm.core.Argument object at 0x10afbdb50>, _llvm=<llvm.core.Argument object at 0x10afbdb50>, typ='f64')>, arg2 = <Variable(val=<llvm.core.Argument object at 0x10afbdb50>, _llvm=<llvm.core.Argument object at 0x10afbdb50>, typ='f64')>
resolve_type() ==> 'f64'
Warning: Could not create fast version... 'module' object has no attribute 'PASS_DEAD_CODE_ELIMINATION'
Traceback (most recent call last):
File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/decorators.py", line 55, in vectorize
t.translate()
File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/numba/translate.py", line 935, in translate
fpm.add(lp.PASS_DEAD_CODE_ELIMINATION)
AttributeError: 'module' object has no attribute 'PASS_DEAD_CODE_ELIMINATION'

Note that I had to install minivect by manually cloning from numba/minivect. I don't know if that's causing the problem.

Minivect submodule not installing properly

➜ ~ git clone git://github.com/numba/numba.git
Cloning into 'numba'...
remote: Counting objects: 1130, done.
remote: Compressing objects: 100% (450/450), done.
remote: Total 1130 (delta 713), reused 1080 (delta 666)
Receiving objects: 100% (1130/1130), 2.27 MiB | 1.42 MiB/s, done.
Resolving deltas: 100% (713/713), done.
➜ ~ cd numba
➜ numba git:(master)) git submodule init
Submodule 'numba/minivect' ([email protected]:numba/minivect.git) registered for path 'numba/minivect'
➜ numba git:(master)) git submodule update
Cloning into 'numba/minivect'...
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
Clone of '[email protected]:numba/minivect.git' into submodule path 'numba/minivect' failed

Manually cloning into numba/numba/minivect allows numba to compile and import properly. However, I'm still unable to @vectorize functions.

Error calling functions with no arguments

See

https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/Jg1Un5bflOQ

Support for asserts

Currently asserts seems to be not supported and exception of missing visit_assert method is thrown.