Unable to build: Python include path missing about lab HOT 30 CLOSED

The includes path is relative to /usr (given by the path value in WORKSPACE), so it should start with local/lib/... in your case.

from lab.

Ah, OK, I think I had this problem too on some distributions. Try amending your python.BUILD to include the Python paths, like so:

# Description:
#   Build rule for Python and Numpy.
#   This rule works for Debian and Ubuntu. Other platforms might keep the
#   headers in different places, cf. 'How to build DeepMind Lab' in build.md.

cc_library(
    name = "python",
    hdrs = glob([
        "include/python2.7/*.h",
         "lib64/python2.7/site-packages/numpy/core/include/**/*.h",
    ]),
    includes = [
        "include/python2.7",
        "lib64/python2.7/site-packages/numpy/core/include",
    ],
    visibility = ["//visibility:public"],
)

(Use locate arrayobject.h) to find the actual paths on your system.)

from lab.

Hey, did you figure out how to solve it? I'm having this error when I tried to build it:

python/dmlab_module.c:29:31: fatal error: numpy/arrayobject.h: No such file or directory

Did you come across error message?

from lab.

Did you install the Numpy development package? See https://github.com/deepmind/lab/blob/master/docs/build.md for details.

from lab.

Yeah, I've the numpy-development package ( and the updated versions of other packages specified on the link too )

from lab.

Sorry, the reference was unclear. My question about numpy-dev was for @jingweiz; @hercky, I don't know what's going on with you, it looks like some environmental problem. Is that a normal, local desktop machine, not some kind of virtual or remote one?

from lab.

Yeah, a normal local desktop machine .

from lab.

@hercky: Can you try bazel --expunge and then try again?

from lab.

@hercky: Any updates on this?

from lab.

Hi @tkoeppe

I guess you meant the command bazel clean --expunge. I tried running that but it too fails with the error:

WARNING: Output base '/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b' is on NFS. This may lead to surprising failures and undetermined behavior.
.
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
ERROR: /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/server/.nfs0000000001ea365500000a0e (Device or resource busy).

from lab.

I found a similar issue at tensorflow/tensorflow#3652 and bazelbuild/bazel#2033 . Using the hints from them, I was able to run the clean

bazel --output_base=/tmp/bazel/lab clean --expunge

successfully. But the error on running bazel --output_base=/tmp/bazel/lab run :game -- --level_script tests/demo_map --verbose_failures is still present (although this time it doesn't fails immediately and downloads some resources first). The error message now is:

........
INFO: Found 1 target...
ERROR: /home/ml/hsatij/code/libs/lab/deepmind/support/BUILD:37:1: C++ compilation of rule '//deepmind/support:stringprintf' failed: linux-sandbox failed: error executing command /tmp/bazel/lab/execroot/lab/_bin/linux-sandbox @/tmp/bazel/lab/bazel-sandbox/f0413033-1321-4272-8da6-d30340a1617e-0/linux-sandbox.params -- /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' ... (remaining 34 argument(s) skipped).
src/main/tools/linux-sandbox-pid1.cc:398: "remount(NULL, /tmp/bazel/lab/bazel-sandbox/f0413033-1321-4272-8da6-d30340a1617e-0/tmp/home/ndg/projects, NULL, 2101281, NULL)": No such file or directory
Target //:game failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 33.670s, Critical Path: 6.86s
ERROR: Build failed. Not running target.

from lab.

I am having a similar problem with you. The error message I get after executing bazel --output_base=/tmp/bazel/lab run :game -- --level_script tests/demo_map --verbose_failures was:

........
INFO: Found 1 target...
ERROR: /home/ck1/fyp/lab/deepmind/support/BUILD:37:1: C++ compilation of rule '//deepmind/support:stringprintf' failed: Process exited with status 1 [sandboxed].
src/main/tools/linux-sandbox-pid1.cc:193: "mount(/tmp, /tmp, NULL, MS_BIND, NULL)": Invalid argument
Use --strategy=CppCompile=standalone to disable sandboxing for the failing actions.
Target //:game failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 124.174s, Critical Path: 0.13s
ERROR: Build failed. Not running target.

Which is way too esoteric for me as a rookie to bazel.

Any help would be appreciated.

from lab.

What do you have at line 74 in the file below?
ERROR: /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/external/jpeg_archive/BUILD:74:1

from lab.

I meet a error like this.Do you know how to solve it.
bazel-out/local-fastbuild/bin/deepmind/lua/_objs/table_ref/deepmind/lua/table_ref.pic.o:table_ref.cc:function deepmind::lab::lua::TableRef::TableRef(deepmind::lab::lua::TableRef const&): error: undefined reference to 'luaL_ref'
bazel-out/local-fastbuild/bin/deepmind/lua/_objs/table_ref/deepmind/lua/table_ref.pic.o:table_ref.cc:function deepmind::lab::lua::TableRef::KeyCount() const: error: undefined reference to 'lua_pushnil'
bazel-out/local-fastbuild/bin/deepmind/lua/_objs/table_ref/deepmind/lua/table_ref.pic.o:table_ref.cc:function deepmind::lab::lua::TableRef::KeyCount() const: error: undefined reference to 'lua_next'
bazel-out/local-fastbuild/bin/deepmind/lua/_objs/table_ref/deepmind/lua/table_ref.pic.o:table_ref.cc:function deepmind::lab::lua::TableRef::Create(lua_State*): error: undefined reference to 'luaL_ref'
bazel-out/local-fastbuild/bin/deepmind/lua/_objs/table_ref/deepmind/lua/table_ref.pic.o:table_ref.cc:function deepmind::lab::lua::Read(lua_State*, int, deepmind::lab::lua::TableRef*): error: undefined reference to 'luaL_ref'
collect2: error: ld returned 1 exit status
Use --strategy=CppLink=standalone to disable sandboxing for the failing actions.
Target //:game failed to build
Use --verbose_failures to see the command lines of failed build steps.

from lab.

Is this still a problem?

from lab.

I still have the problem of:

python/dmlab_module.c:29:31: fatal error: numpy/arrayobject.h: No such file or directory

how did you solve them finally?

from lab.

@rockkingjy: Did you install the necessary numpy development package?

from lab.

Yes, I run

sudo pip install numpy

I got:

Requirement already satisfied: numpy in /usr/local/lib/python2.7/dist-packages

from lab.

No, I mean the system development package. E.g. apt-get install python-numpy on Debianesque systems. (We need the C headers, not just the Python package.)

from lab.

Hi~ Thanks for you answer, but I think yes:

sudo apt-get install python-numpy

and have:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
python-numpy is already the newest version (1:1.11.0-1ubuntu1).

And other informations:

$ python --version
Python 2.7.12
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial

from lab.

Thanks, yes, I've done:

locate /arrayobject.h

that gave:

/usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy/arrayobject.h

And I've changed the python.BUILD to:

cc_library(
    name = "python",
    hdrs = glob([
        "include/python2.7/*.h",
        "lib/python2.7/dist-packages/numpy/core/include/**/*.h",
    ]),
    includes = [
        "include/python2.7",
        "lib/python2.7/dist-packages/numpy/core/include",
    ],
    visibility = ["//visibility:public"],
)

But still the same problem:

$ sudo bazel build :deepmind_lab.so --define headless=glx
INFO: Analysed target //:deepmind_lab.so (1 packages loaded).
INFO: Found 1 target...
ERROR: /home/enroutelab/Amy/lab/BUILD:988:1: C++ compilation of rule '//:deepmind_lab.so' failed (Exit 1)
python/dmlab_module.c:29:31: fatal error: numpy/arrayobject.h: No such file or directory
compilation terminated.

from lab.

Aha, it seems that it works, great~ thanks so much!

from lab.

No problem :-) Perhaps we should add a note to the documentation that this may be required.

from lab.

Oh, we already have that note in https://github.com/deepmind/lab/blob/master/docs/build.md :-S It's Item 5 of https://github.com/deepmind/lab/blob/master/docs/build.md#building-on-suse-linux.

from lab.

Oh, yes, but for my case of ubuntu, I haven't notice this... because it's in "Build on SUSE Linux subtitle"...

from lab.

(Perhaps a documentation update would be in order.)

from lab.

I had this problem as well on Ubuntu 16.04 and @tkoeppe comment above provided me enough info to edit my Python.BUILD accordingly and now I am past this problem. Thanks!

from lab.

@MrDadaGuy: Ubuntu 16 is what's running the Travis CI test, I believe, so you can always look at that one to see a worked example. I would hope that Ubunutu 16 doesn't require any of the editing that we need on Suse, though. (Scratch that, Travis is running Ubuntu 14 at the moment.)

from lab.

Note also that you need to make sure that the Python and Numpy devel libraries you've compiled against match the Python binary that is found when you just say python. This seems straight-forward if you are sudoing and can install libraries system-wide, but it has bitten me when I tried homebrew on macOS and there were different Python versions around.

from lab.

I think we can close this issue. Feel free to reopen if there is new information.

from lab.