Tests failing with Bazel 0.2.3 and Tensorflow 0.9 about magenta HOT 20 CLOSED

Thanks both of you! Upgrading my system python to 2.7.11 made all the tests pass!

from magenta.

@act65 try condo install --channel https://conda.anaconda.org/conda-forge protobuf=3.0.0b2.post2.

from magenta.

I'm running into the same error, can't import tensorflow

If you type python then import tensorflow as tf and get this error, then I think this is an issue with setting up tensorflow for use in your python environment, and doesn't necessarily have a thing to do with magenta, except for the fact that magenta depends on this working in some way.

I tried installing tensorflow for python 2.7/osx with pip, but got the error "tensorflow-0.9.0rc0-py2-none-any.whl is not a supported wheel on this platform.". I've now solved this by updating my python from 2.7.10 to 2.7.11! My bazel tests are now passing.

Before thinking to update python, I tried using the python 3/osx tensorflow package, and encountered the same error, "can't import tensorflow". Surely magenta supports python 3, the problem is making python be python 3 and not 2.7. I tried aliasing python to python3, but that didn't work. Creating a symlink could work, but I'm going to let others explore that right now: https://www.google.com/search?q=change+default+python+to+python3

I also tried tensorflows docker image but python isn't even setup inside their docker container, perhaps it should, but at that point I stopped and focused on other methods.

from magenta.

I ran into similar issues and assumed it was PEBKAC. FWIW I could not get it to go with magenta either.

from magenta.

I finally got things to work with python 2.7. The secret sauce for me was to completely remove python 3, reinstall python 2 from scratch ensuring the brew postinstall worked, then do

brew unlink python && brew link --overwrite python
pip install --upgrade pip
pip install --upgrade setuptools

After that I followed the tensorflow setup instructions, and everything worked!

Hopefully that helps others.

from magenta.

I believe I'm having a similar issue when running the tests. I'm on:

OS X 10.11.5, Anaconda Python 2.7.11, Bazel 0.2.3, Tensorflow 0.9 with GPU, Cuda 7.5, CuDNN 5.1 RC

the error I get is:

Traceback (most recent call last): File "/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/__main__/magenta/lib/midi_io_test.py", line 27, in <module> import tensorflow as tf File "/Users/Atom/.anaconda/lib/python2.7/site-packages/tensorflow/__init__.py", line 23, in <module> from tensorflow.python import * File "/Users/Atom/.anaconda/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 48, in <module> from tensorflow.python import pywrap_tensorflow File "/Users/Atom/.anaconda/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module> _pywrap_tensorflow = swig_import_helper() File "/Users/Atom/.anaconda/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: dlopen(/Users/Atom/.anaconda/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so, 10): Library not loaded: @rpath/libcudart.7.5.dylib Referenced from: /Users/Atom/.anaconda/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so Reason: image not found

which seems to be caused by missing environment variables that point to the GPU libraries (tensorflow/tensorflow#2278).

Indeed, when I added import os; print os.environ to one of the tested python files, the resulting dictionary did not contain DYLD_LIBRARY_PATH. Why is the environment different? Is there a way to keep / include a user's environment variables?

For reference here's the printed os.environ from the test:

{'PYTHON_RUNFILES': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles', 'TZ': 'UTC', 'PYTHONPATH': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles:/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/protobuf/python:/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/__main__:/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/midi:/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/pretty_midi:/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/protobuf:/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/six_archive', 'VERSIONER_PYTHON_VERSION': '2.7', 'TEST_TIMEOUT': '300', 'XML_OUTPUT_FILE': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/testlogs/magenta/midi_io_test/test.xml', 'JAVA_RUNFILES': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles', 'VERSIONER_PYTHON_PREFER_32_BIT': 'no', 'TEST_WORKSPACE': '__main__', 'TEST_SRCDIR': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles', '__CF_USER_TEXT_ENCODING': '0x1F6:0x0:0x0', 'SHLVL': '1', 'PWD': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/bazel-out/local-opt/bin/magenta/midi_io_test.runfiles/__main__', '_': 'magenta/midi_io_test', 'TEST_SIZE': 'medium', 'TEST_TMPDIR': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/_tmp/midi_io_test_6', 'PATH': '.:/usr/local/cuda/bin:/Users/Atom/.anaconda/bin:/Library/TeX/Distributions/TeXLive-2014-Basic.texdist/Contents/Programs/x86_64:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin:/opt/X11/bin:/usr/local/git/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin:/usr/texbin', 'GTEST_TMP_DIR': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta/_tmp/midi_io_test_6', 'OLDPWD': '/private/var/tmp/_bazel_Atom/cc11e662fbaa4b129c6842f2a973b85f/magenta', 'TMPDIR': '/var/folders/h0/j056qfyd7_s0_n1hj2z_716h0000gp/T/'}

and from a regular ipython session:

{'LESS': '-R', 'LC_CTYPE': 'en_CA.UTF-8', 'TERM_PROGRAM_VERSION': '361.1', 'LOGNAME': 'Atom', 'USER': 'Atom', 'PATH': '/usr/local/cuda/bin:/Users/Atom/.anaconda/bin:/Library/TeX/Distributions/TeXLive-2014-Basic.texdist/Contents/Programs/x86_64:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin:/opt/X11/bin:/usr/local/git/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin:/usr/texbin', 'HOME': '/Users/Atom', 'DISPLAY': '/private/tmp/com.apple.launchd.Tc0rYXUYhi/org.macosforge.xquartz:0', 'TERM_PROGRAM': 'Apple_Terminal', 'LANG': 'en_CA.UTF-8', 'TERM': 'xterm-256color', 'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.y4UoBj0q7B/Render', 'SHLVL': '2', 'XPC_FLAGS': '0x0', '_': '/Users/Atom/.anaconda/python.app/Contents/MacOS/python', 'TERM_SESSION_ID': '9C37E199-5885-47D0-90FC-5599E0F31044', 'XPC_SERVICE_NAME': '0', 'CUDA_HOME': '/usr/local/cuda', 'PYTHONPATH': '/Users/Atom/Dropbox/Aercoustics/ModCalcTools/Aelpy Library/source:', 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.dBG4xDDrvt/Listeners', 'DYLD_LIBRARY_PATH': ':/usr/local/cuda/lib', 'SHELL': '/bin/zsh', 'TMPDIR': '/var/folders/h0/j056qfyd7_s0_n1hj2z_716h0000gp/T/', 'LSCOLORS': 'Gxfxcxdxbxegedabagacad', 'PYTHONEXECUTABLE': '/Users/Atom/.anaconda/bin/python', '__CF_USER_TEXT_ENCODING': '0x1F6:0x0:0x0', 'PWD': '/Users/Atom/Dropbox/Year4/ECE521/Reference/tensorflow/magenta', 'PAGER': 'less'}

from magenta.

@nathanielatom , which python version are you using?

from magenta.

@cinjon, just updated my comment, sorry about that.

from magenta.

@nathanielatom, I've usually seen that error when libcudart.7.5 wasn't copied to the correct cuda installation. Would you mind following the installation instructions one more time just to make sure that isn't it?

from magenta.

Hey,
mine doesnt work either. Mac OSX 10.11, Anaconda 3.5, Bazel 0.3, Tensorflow 0.9. And just cloned this repo.

$ bazel test //magenta/...
ERROR: package contains errors: magenta/magenta/protobuf.
ERROR: error loading package 'magenta/magenta/protobuf': Encountered error while reading extension file 'protobuf.bzl': no such package '@protobuf//': error loading package 'external': The repository named 'protobuf' could not be resolved.
INFO: Elapsed time: 0.226s
ERROR: Couldn't start the build. Unable to run tests.

Cant be bothered going back to python 2.7, will just wait for fix/help.

from magenta.

@cinjon I ran

wget $CUDNN_URL
tar -xzf cudnn-*-osx-x64-v*.tgz
sudo mv cuda/include/cudnn.h /Developer/NVIDIA/CUDA-7.5/include/
sudo mv cuda/lib/libcudnn* /Developer/NVIDIA/CUDA-7.5/lib
sudo ln -s /Developer/NVIDIA/CUDA-7.5/lib/libcudnn* /usr/local/cuda/lib/

and my environment contains

export CUDA_HOME="/usr/local/cuda"
export DYLD_LIBRARY_PATH="$DYLD_LIBRARY_PATH:$CUDA_HOME/lib"

and ls /usr/local/cuda/lib | grep "libcudart.7.5" gives libcudart.7.5.dylib.

Tensorflow 0.9 itself works great with GPU.

from magenta.

@act65 your issue sounds like something different. Can you open a separate bug?

@nathanielatom do the tests work if you first do a 'bazel build' on the test target and then run the test directly? The result of bazel build should include the bazel-bin/ location of the test file.

If that works, the issue is that bazel strips out a lot of environment variables when it runs things, in an effort to keep a hermetic build/test environment. We're kind of working around things by relying on an external tensorflow installation.

In the short term, you could just run things directly, instead of using bazel build/test. Longer term, we're working on ways to depend directly on tensorflow and have it correctly autodetect whether your system can support a GPU. Some more details here: tensorflow/tensorflow#2873

from magenta.

I also have some problem on running test. There are 2 fail on 8 test item.

INFO: Found 39 targets and 8 test targets...
FAIL: //magenta/lib:midi_io_test (see /private/var/tmp/_bazel_kwangin/001031a49558f53b54e889be1a87c437/magenta/bazel-out/local_darwin-opt/testlogs/magenta/lib/midi_io_test/test.log).
FAIL: //magenta/scripts:convert_midi_dir_to_note_sequences_test (see /private/var/tmp/_bazel_kwangin/001031a49558f53b54e889be1a87c437/magenta/bazel-out/local_darwin-opt/testlogs/magenta/scripts/convert_midi_dir_to_note_sequences_test/test.log).
INFO: Elapsed time: 8.243s, Critical Path: 2.52s
//magenta/lib:melodies_lib_test                                 (cached) PASSED in 2.2s
//magenta/lib:note_sequence_io_test                             (cached) PASSED in 1.8s
//magenta/lib:sequences_lib_test                                (cached) PASSED in 1.3s
//magenta/models/shared:melody_rnn_create_dataset_test          (cached) PASSED in 2.4s
//magenta/pipelines:pipeline_test                               (cached) PASSED in 3.4s
//magenta/pipelines:pipelines_common_test                       (cached) PASSED in 3.2s
//magenta/lib:midi_io_test                                               FAILED in 0.2s
  /private/var/tmp/_bazel_kwangin/001031a49558f53b54e889be1a87c437/magenta/bazel-out/local_darwin-opt/testlogs/magenta/lib/midi_io_test/test.log
//magenta/scripts:convert_midi_dir_to_note_sequences_test                FAILED in 1.7s
  /private/var/tmp/_bazel_kwangin/001031a49558f53b54e889be1a87c437/magenta/bazel-out/local_darwin-opt/testlogs/magenta/scripts/convert_midi_dir_to_note_sequences_test/test.log

Executed 2 out of 8 tests: 6 tests pass and 2 fail locally.

Both log has comment as 'ImportError: No module named midi'.

Traceback (most recent call last):
  File "/private/var/tmp/_bazel_kwangin/001031a49558f53b54e889be1a87c437/magenta/bazel-out/local_darwin-opt/bin/magenta/lib/midi_io_test.runfiles/magenta/lib/midi_io_test.py", line 21, in <module>
    import pretty_midi
  File "/private/var/tmp/_bazel_kwangin/001031a49558f53b54e889be1a87c437/magenta/bazel-out/local_darwin-opt/bin/magenta/lib/midi_io_test.runfiles/external/pretty_midi/pretty_midi.py", line 6, in <module>
    import midi
ImportError: No module named midi

Does anyone know about this?

from magenta.

Hi, also having similar problem running the initial test.
(python 2.7.12 / tensorflow 0.9.0 / OSX 10.11.6)

Executing the following code from the instruction,
bazel test //magenta/...

I get these error messages. :

WARNING: /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/external/protobuf/WORKSPACE:1: Workspace name in /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/external/protobuf/WORKSPACE (@__main__) does not match the name given in the repository's definition (@protobuf); this will cause a build error in future versions.
INFO: Found 45 targets and 10 test targets...
FAIL: //magenta/lib:note_sequence_io_test (see /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/lib/note_sequence_io_test/test.log).
FAIL: //magenta/pipelines:pipeline_test (see /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/pipelines/pipeline_test/test.log).
FAIL: //magenta/scripts:convert_midi_dir_to_note_sequences_test (see /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/scripts/convert_midi_dir_to_note_sequences_test/test.log).
INFO: Elapsed time: 1.181s, Critical Path: 0.76s
//magenta/lib:melodies_lib_test                                 (cached) PASSED in 1.1s
//magenta/lib:midi_io_test                                      (cached) PASSED in 2.1s
//magenta/lib:sequences_lib_test                                (cached) PASSED in 1.0s
//magenta/models/shared:melody_rnn_create_dataset_test          (cached) PASSED in 1.0s
//magenta/pipelines:dag_pipeline_test                           (cached) PASSED in 1.0s
//magenta/pipelines:pipelines_common_test                       (cached) PASSED in 1.0s
//magenta/pipelines:statistics_test                             (cached) PASSED in 1.0s
//magenta/lib:note_sequence_io_test                                      FAILED in 0.6s
  /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/lib/note_sequence_io_test/test.log
//magenta/pipelines:pipeline_test                                        FAILED in 0.7s
  /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/pipelines/pipeline_test/test.log
//magenta/scripts:convert_midi_dir_to_note_sequences_test                FAILED in 0.8s
  /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/scripts/convert_midi_dir_to_note_sequences_test/test.log

Executed 3 out of 10 tests: 7 tests pass and 3 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Could anyone give me an advice?

With the suggested option, here are the error messages.

WARNING: /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/external/protobuf/WORKSPACE:1: Workspace name in /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/external/protobuf/WORKSPACE (@__main__) does not match the name given in the repository's definition (@protobuf); this will cause a build error in future versions.
INFO: Found 45 targets and 10 test targets...
FAIL: //magenta/lib:note_sequence_io_test (see /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/lib/note_sequence_io_test/test.log).
FAIL: //magenta/pipelines:pipeline_test (see /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/pipelines/pipeline_test/test.log).
FAIL: //magenta/scripts:convert_midi_dir_to_note_sequences_test (see /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/scripts/convert_midi_dir_to_note_sequences_test/test.log).
INFO: Elapsed time: 1.193s, Critical Path: 0.74s
//magenta/lib:melodies_lib_test                                 (cached) PASSED in 1.1s
  WARNING: //magenta/lib:melodies_lib_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/lib:midi_io_test                                      (cached) PASSED in 2.1s
  WARNING: //magenta/lib:midi_io_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/lib:sequences_lib_test                                (cached) PASSED in 1.0s
  WARNING: //magenta/lib:sequences_lib_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/models/shared:melody_rnn_create_dataset_test          (cached) PASSED in 1.0s
  WARNING: //magenta/models/shared:melody_rnn_create_dataset_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/pipelines:dag_pipeline_test                           (cached) PASSED in 1.0s
  WARNING: //magenta/pipelines:dag_pipeline_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/pipelines:pipelines_common_test                       (cached) PASSED in 1.0s
  WARNING: //magenta/pipelines:pipelines_common_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/pipelines:statistics_test                             (cached) PASSED in 1.0s
  WARNING: //magenta/pipelines:statistics_test: Test execution time (0.0s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
//magenta/lib:note_sequence_io_test                                      FAILED in 0.6s
  /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/lib/note_sequence_io_test/test.log
//magenta/pipelines:pipeline_test                                        FAILED in 0.7s
  /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/pipelines/pipeline_test/test.log
//magenta/scripts:convert_midi_dir_to_note_sequences_test                FAILED in 0.7s
  /private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/testlogs/magenta/scripts/convert_midi_dir_to_note_sequences_test/test.log

Executed 3 out of 10 tests: 7 tests pass and 3 fail locally.

And here are the 3 test.log files.

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
.E.
======================================================================
ERROR: testNoteSequenceRecordWriterAndIterator (__main__.NoteSequenceIoTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/bin/magenta/lib/note_sequence_io_test.runfiles/__main__/magenta/lib/note_sequence_io_test.py", line 52, in testNoteSequenceRecordWriterAndIterator
    writer.write(sequence)
AttributeError: 'NoneType' object has no attribute 'write'

----------------------------------------------------------------------
Ran 3 tests in 0.010s

FAILED (errors=1)

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/bin/magenta/scripts/convert_midi_dir_to_note_sequences_test.runfiles/__main__/magenta/scripts/convert_midi_dir_to_note_sequences_test.py", line 23, in <module>
    from magenta.scripts import convert_midi_dir_to_note_sequences
  File "/private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/bin/magenta/scripts/convert_midi_dir_to_note_sequences_test.runfiles/__main__/magenta/scripts/convert_midi_dir_to_note_sequences.py", line 40, in <module>
    tf.app.flags.DEFINE_bool('recursive', False,
AttributeError: 'module' object has no attribute 'DEFINE_bool'

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
EE.........
======================================================================
ERROR: testFileIteratorNotRecursive (__main__.PipelineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/bin/magenta/pipelines/pipeline_test.runfiles/__main__/magenta/pipelines/pipeline_test.py", line 89, in testFileIteratorNotRecursive
    tf.gfile.MakeDirs(os.path.dirname(abs_path))
  File "/Library/Python/2.7/site-packages/tensorflow/python/platform/default/_gfile.py", line 294, in MakeDirs
    os.makedirs(path, mode)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/folders/ft/pglvm7vd2z314vmrkkrqgsc40000gn/T/pipeline_test/tmpXijx5Y'

======================================================================
ERROR: testFileIteratorRecursive (__main__.PipelineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_kaka/d3542784be43f5a4052aba9655d0cf38/magenta/bazel-out/local-opt/bin/magenta/pipelines/pipeline_test.runfiles/__main__/magenta/pipelines/pipeline_test.py", line 65, in testFileIteratorRecursive
    tf.gfile.MakeDirs(os.path.dirname(abs_path))
  File "/Library/Python/2.7/site-packages/tensorflow/python/platform/default/_gfile.py", line 294, in MakeDirs
    os.makedirs(path, mode)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/folders/ft/pglvm7vd2z314vmrkkrqgsc40000gn/T/pipeline_test/tmpviHWZd'

----------------------------------------------------------------------
Ran 11 tests in 0.042s

FAILED (errors=2)

from magenta.

For folks still having build errors, Bazel 0.3.1 was released recently and seems to clear up some issues. Can you give it a try and see if you're still getting errors? https://github.com/bazelbuild/bazel/releases

from magenta.

@cghawthorne Now it works. Test are all passed. Thanks~
I tested on Python 2.7.10, Bazel 0.3.1
Hope other people works.

from magenta.

I'm going to close this issue out since we no longer support TensorFlow 0.10. Please open a new issue if problems persist after you upgrade.

from magenta.

@nathanielatom
Ubuntu 14.04 Python2.7 bazel 0.4.4, error when test magenta:
jcc@jcc:~/magenta$ bazel test //magenta:all
ERROR: /home/jcc/.cache/bazel/_bazel_jcc/ed5599de3378c6e0a5a71ebe1650713e/external/protobuf/protobuf.bzl:91:19: name 'HOST_CFG' is not defined.
ERROR: /home/jcc/magenta/magenta/BUILD:21:1: error loading package 'magenta/protobuf': Extension 'protobuf.bzl' has errors and referenced by '//magenta:magenta'.
ERROR: Analysis of target '//magenta:magenta' failed; build aborted.
INFO: Elapsed time: 0.155s
ERROR: Couldn't start the build. Unable to run tests.
Could you please give some advice?

from magenta.

sudo bazel test //magenta/...
Password:
Extracting Bazel installation...
.........
INFO: Found 136 targets and 39 test targets...
INFO: From Compiling external/protobuf/src/google/protobuf/util/time_util.cc [for host]:
external/protobuf/src/google/protobuf/util/time_util.cc:52:18: warning: unused variable 'kMicrosPerMillisecond' [-Wunused-const-variable]
static const int kMicrosPerMillisecond = 1000;
^
external/protobuf/src/google/protobuf/util/time_util.cc:56:19: warning: unused variable 'kTimestampFormat' [-Wunused-const-variable]
static const char kTimestampFormat[] = "%E4Y-%m-%dT%H:%M:%S";
^
2 warnings generated.
INFO: From Compiling external/protobuf/src/google/protobuf/compiler/js/embed.cc [for host]:
external/protobuf/src/google/protobuf/compiler/js/embed.cc:37:12: warning: unused variable 'output_file' [-Wunused-const-variable]
const char output_file[] = "well_known_types_embed.cc";
^
1 warning generated.
INFO: From Linking external/protobuf/libprotobuf.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/host/bin/external/protobuf/_objs/protobuf/external/protobuf/src/google/protobuf/io/gzip_stream.o has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/host/bin/external/protobuf/_objs/protobuf/external/protobuf/src/google/protobuf/util/internal/error_listener.o has no symbols
INFO: From Linking external/protobuf/libprotobuf_lite.a [for host]:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/host/bin/external/protobuf/_objs/protobuf_lite/external/protobuf/src/google/protobuf/stubs/atomicops_internals_x86_gcc.o has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/host/bin/external/protobuf/_objs/protobuf_lite/external/protobuf/src/google/protobuf/stubs/atomicops_internals_x86_msvc.o has no symbols
FAIL: //magenta/common:concurrency_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/common/concurrency_test/test.log).
FAIL: //magenta/models/improv_rnn:improv_rnn_create_dataset_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/improv_rnn/improv_rnn_create_dataset_test/test.log).
FAIL: //magenta/common:state_util_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/common/state_util_test/test.log).
FAIL: //magenta/models/melody_rnn:melody_rnn_create_dataset_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/melody_rnn/melody_rnn_create_dataset_test/test.log).
FAIL: //magenta/models/shared:events_rnn_graph_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/shared/events_rnn_graph_test/test.log).
FAIL: //magenta/models/polyphony_rnn:polyphony_encoder_decoder_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/polyphony_rnn/polyphony_encoder_decoder_test/test.log).
FAIL: //magenta/models/polyphony_rnn:polyphony_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/polyphony_rnn/polyphony_lib_test/test.log).
FAIL: //magenta/models/polyphony_rnn:polyphony_rnn_create_dataset_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/polyphony_rnn/polyphony_rnn_create_dataset_test/test.log).
FAIL: //magenta/models/drums_rnn:drums_rnn_create_dataset_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/drums_rnn/drums_rnn_create_dataset_test/test.log).
FAIL: //magenta/models/pianoroll_rnn_nade:pianoroll_rnn_nade_create_dataset_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/pianoroll_rnn_nade/pianoroll_rnn_nade_create_dataset_test/test.log).
FAIL: //magenta/scripts:convert_dir_to_note_sequences_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/scripts/convert_dir_to_note_sequences_test/test.log).
FAIL: //magenta/models/rl_tuner:rl_tuner_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/rl_tuner/rl_tuner_test/test.log).
INFO: From Compiling external/protobuf/src/google/protobuf/util/time_util.cc:
external/protobuf/src/google/protobuf/util/time_util.cc:52:18: warning: unused variable 'kMicrosPerMillisecond' [-Wunused-const-variable]
static const int kMicrosPerMillisecond = 1000;
^
external/protobuf/src/google/protobuf/util/time_util.cc:56:19: warning: unused variable 'kTimestampFormat' [-Wunused-const-variable]
static const char kTimestampFormat[] = "%E4Y-%m-%dT%H:%M:%S";
^
2 warnings generated.
FAIL: //magenta/interfaces/midi:midi_hub_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/interfaces/midi/midi_hub_test/test.log).
FAIL: //magenta/pipelines:statistics_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/statistics_test/test.log).
FAIL: //magenta/pipelines:pipelines_common_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/pipelines_common_test/test.log).
FAIL: //magenta/pipelines:pipeline_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/pipeline_test/test.log).
FAIL: //magenta/pipelines:melody_pipelines_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/melody_pipelines_test/test.log).
FAIL: //magenta/pipelines:lead_sheet_pipelines_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/lead_sheet_pipelines_test/test.log).
FAIL: //magenta/pipelines:drum_pipelines_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/drum_pipelines_test/test.log).
FAIL: //magenta/pipelines:dag_pipeline_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/dag_pipeline_test/test.log).
FAIL: //magenta/pipelines:chord_pipelines_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/pipelines/chord_pipelines_test/test.log).
FAIL: //magenta/music:sequences_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/sequences_lib_test/test.log).
FAIL: //magenta/music:sequence_generator_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/sequence_generator_test/test.log).
FAIL: //magenta/music:pianoroll_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/pianoroll_lib_test/test.log).
FAIL: //magenta/music:note_sequence_io_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/note_sequence_io_test/test.log).
FAIL: //magenta/music:musicxml_parser_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/musicxml_parser_test/test.log).
FAIL: //magenta/music:pianoroll_encoder_decoder_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/pianoroll_encoder_decoder_test/test.log).
FAIL: //magenta/music:midi_io_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/midi_io_test/test.log).
FAIL: //magenta/music:melody_encoder_decoder_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/melody_encoder_decoder_test/test.log).
FAIL: //magenta/music:musicnet_io_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/musicnet_io_test/test.log).
FAIL: //magenta/music:melodies_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/melodies_lib_test/test.log).
FAIL: //magenta/music:lead_sheets_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/lead_sheets_lib_test/test.log).
FAIL: //magenta/music:events_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/events_lib_test/test.log).
FAIL: //magenta/music:drums_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/drums_lib_test/test.log).
FAIL: //magenta/music:drums_encoder_decoder_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/drums_encoder_decoder_test/test.log).
FAIL: //magenta/music:chords_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/chords_lib_test/test.log).
FAIL: //magenta/music:chords_encoder_decoder_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/chords_encoder_decoder_test/test.log).
FAIL: //magenta/music:encoder_decoder_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/encoder_decoder_test/test.log).
FAIL: //magenta/music:chord_symbols_lib_test (see /private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/music/chord_symbols_lib_test/test.log).
INFO: Elapsed time: 79.245s, Critical Path: 10.22s
//magenta/common:concurrency_test FAILED in 0.2s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/common/concurrency_test/test.log
//magenta/common:state_util_test FAILED in 0.5s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/common/state_util_test/test.log
//magenta/interfaces/midi:midi_hub_test FAILED in 0.2s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/interfaces/midi/midi_hub_test/test.log
//magenta/models/drums_rnn:drums_rnn_create_dataset_test FAILED in 0.2s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/drums_rnn/drums_rnn_create_dataset_test/test.log
//magenta/models/improv_rnn:improv_rnn_create_dataset_test FAILED in 0.1s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/improv_rnn/improv_rnn_create_dataset_test/test.log
//magenta/models/melody_rnn:melody_rnn_create_dataset_test FAILED in 0.1s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/melody_rnn/melody_rnn_create_dataset_test/test.log
//magenta/models/pianoroll_rnn_nade:pianoroll_rnn_nade_create_dataset_test FAILED in 0.2s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/pianoroll_rnn_nade/pianoroll_rnn_nade_create_dataset_test/test.log
//magenta/models/polyphony_rnn:polyphony_encoder_decoder_test FAILED in 0.2s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/polyphony_rnn/polyphony_encoder_decoder_test/test.log
//magenta/models/polyphony_rnn:polyphony_lib_test FAILED in 0.2s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlogs/magenta/models/polyphony_rnn/polyphony_lib_test/test.log
//magenta/models/polyphony_rnn:polyphony_rnn_create_dataset_test FAILED in 0.1s
/private/var/tmp/_bazel_root/f7b8a482e96a40315508e7c5aab958fc/execroot/magenta/bazel-out/darwin_x86_64-opt/testlog

from magenta.

@neelkadia can you please file a new issue? I suspect your problem is not related to the issue in this bug. When you create a new issue, please include the errors in one of the log files from the output.

from magenta.