Comments (7)
malcolmreynolds
commented on August 19, 2024
2
@Tutufa glad you got it sorted - As far as I'm aware we're not specifying versions of GCC anywhere in the Sonnet repository, but depending on what is installed on your system bazel may pick up mismatching versions. We'll be updating the install instructions soon to try to cover the issues that people have been having, thanks for providing this information.
from sonnet.
malcolmreynolds
commented on August 19, 2024
Thanks for the report. Can you try running the unit tests by doing bazel test sonnet/python:all in the root of the repository? That will give us a bit of a clearer idea of what is happening.
from sonnet.
Tutufa
commented on August 19, 2024
bazel test sonnet/python:all
WARNING: /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/external/org_tensorflow/tensorflow/workspace.bzl:72:5: tf_repo_name was specified to tf_workspace but is no longer used and will be removed in the future.
WARNING: /home/kovalenko/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_library rule //sonnet/python:ops/_resampler_gpu: please do not import '//sonnet/cc/kernels:resampler_op_gpu.cu.cc' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/kovalenko/sonnet/sonnet/tensorflow.bzl:98:16.
WARNING: /home/kovalenko/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_binary rule //sonnet/python:ops/_resampler.so: please do not import '//sonnet/cc:ops/resampler.cc' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/kovalenko/sonnet/sonnet/tensorflow.bzl:108:25.
WARNING: /home/kovalenko/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_binary rule //sonnet/python:ops/_resampler.so: please do not import '//sonnet/cc/kernels:resampler_op.cc' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/kovalenko/sonnet/sonnet/tensorflow.bzl:108:25.
WARNING: /home/kovalenko/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_binary rule //sonnet/python:ops/_resampler.so: please do not import '//sonnet/cc/kernels:resampler_op.h' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/kovalenko/sonnet/sonnet/tensorflow.bzl:108:25.
INFO: Found 14 targets and 25 test targets...
INFO: From ProtoCompile external/org_tensorflow/tensorflow/core/framework/op_gen_overrides.pb.cc:
bazel-out/local-fastbuild/genfiles/external/protobuf/src: warning: directory does not exist.
bazel-out/local-fastbuild/genfiles/external/protobuf/src: warning: directory does not exist.
INFO: From Compiling external/org_tensorflow/tensorflow/core/framework/allocator.cc:
external/org_tensorflow/tensorflow/core/framework/allocator.cc:113:12: warning: 'tensorflow::Allocator* tensorflow::{anonymous}::MakeCpuAllocator()' defined but not used [-Wunused-function]
Allocator* MakeCpuAllocator() {
^
FAIL: //sonnet/python:mlp_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/mlp_test/test.log).
FAIL: //sonnet/python:batch_norm_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/batch_norm_test/test.log).
FAIL: //sonnet/python:convnet_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/convnet_test/test.log).
FAIL: //sonnet/python:util_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/util_test/test.log).
FAIL: //sonnet/python:conv_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/conv_test/test.log).
FAIL: //sonnet/python:alexnet_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/alexnet_test/test.log).
FAIL: //sonnet/python:rnn_core_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/rnn_core_test/test.log).
FAIL: //sonnet/python:basic_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/basic_test/test.log).
FAIL: //sonnet/python:basic_rnn_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/basic_rnn_test/test.log).
FAIL: //sonnet/python:base_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/base_test/test.log).
FAIL: //sonnet/python:block_matrix_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/block_matrix_test/test.log).
FAIL: //sonnet/python:gated_rnn_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/gated_rnn_test/test.log).
FAIL: //sonnet/python:attention_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/attention_test/test.log).
FAIL: //sonnet/python:experimental_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/experimental_test/test.log).
FAIL: //sonnet/python:initializers_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/initializers_test/test.log).
FAIL: //sonnet/python:scale_gradient_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/scale_gradient_test/test.log).
FAIL: //sonnet/python:dilation_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/dilation_test/test.log).
FAIL: //sonnet/python:conv_gpu_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/conv_gpu_test/test.log).
FAIL: //sonnet/python:resampler_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/resampler_test/test.log).
FAIL: //sonnet/python:sequential_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/sequential_test/test.log).
FAIL: //sonnet/python:clip_gradient_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/clip_gradient_test/test.log).
FAIL: //sonnet/python:pondering_rnn_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/pondering_rnn_test/test.log).
FAIL: //sonnet/python:spatial_transformer_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/spatial_transformer_test/test.log).
FAIL: //sonnet/python:embed_test (see /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/embed_test/test.log).
INFO: Elapsed time: 101.583s, Critical Path: 88.86s
//sonnet/python:nest_test PASSED in 1.8s
//sonnet/python:alexnet_test FAILED in 1 out of 2 in 2.2s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/alexnet_test/test.log
//sonnet/python:attention_test FAILED in 1 out of 2 in 1.8s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/attention_test/test.log
//sonnet/python:base_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/base_test/test.log
//sonnet/python:basic_rnn_test FAILED in 1 out of 2 in 2.5s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/basic_rnn_test/test.log
//sonnet/python:basic_test FAILED in 1 out of 2 in 2.4s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/basic_test/test.log
//sonnet/python:batch_norm_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/batch_norm_test/test.log
//sonnet/python:block_matrix_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/block_matrix_test/test.log
//sonnet/python:clip_gradient_test FAILED in 1 out of 2 in 2.1s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/clip_gradient_test/test.log
//sonnet/python:conv_gpu_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/conv_gpu_test/test.log
//sonnet/python:conv_test FAILED in 1 out of 2 in 2.2s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/conv_test/test.log
//sonnet/python:convnet_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/convnet_test/test.log
//sonnet/python:dilation_test FAILED in 1 out of 2 in 2.0s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/dilation_test/test.log
//sonnet/python:embed_test FAILED in 1 out of 2 in 2.0s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/embed_test/test.log
//sonnet/python:experimental_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/experimental_test/test.log
//sonnet/python:gated_rnn_test FAILED in 1 out of 2 in 2.0s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/gated_rnn_test/test.log
//sonnet/python:initializers_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/initializers_test/test.log
//sonnet/python:mlp_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/mlp_test/test.log
//sonnet/python:pondering_rnn_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/pondering_rnn_test/test.log
//sonnet/python:resampler_test FAILED in 1 out of 2 in 2.0s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/resampler_test/test.log
//sonnet/python:rnn_core_test FAILED in 1 out of 2 in 2.3s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/rnn_core_test/test.log
//sonnet/python:scale_gradient_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/scale_gradient_test/test.log
//sonnet/python:sequential_test FAILED in 1 out of 2 in 2.0s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/sequential_test/test.log
//sonnet/python:spatial_transformer_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/spatial_transformer_test/test.log
//sonnet/python:util_test FAILED in 1 out of 2 in 1.9s
/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/util_test/test.log
Executed 25 out of 25 tests: 1 test passes and 24 fail locally.
@malcolmreynolds Also, I checkout r1.1 from git in sonnet/tensorflow and my TF version is 1.1.0
from sonnet.
malcolmreynolds
commented on August 19, 2024
Could you paste in the contents of one of the log files for a test that doesn't use the resampler, eg the contents of /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/pondering_rnn_test/test.log ?
from sonnet.
Tutufa
commented on August 19, 2024
cat /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/execroot/sonnet/bazel-out/local-fastbuild/testlogs/sonnet/python/pondering_rnn_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/bazel-sandbox/9fc9d257-78be-40af-8d1a-aae9a9c04001-430/execroot/sonnet/bazel-out/local-fastbuild/bin/sonnet/python/pondering_rnn_test.runfiles/sonnet/sonnet/python/modules/pondering_rnn_test.py", line 24, in <module>
from sonnet.python.modules import basic_rnn
File "/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/bazel-sandbox/9fc9d257-78be-40af-8d1a-aae9a9c04001-430/execroot/sonnet/bazel-out/local-fastbuild/bin/sonnet/python/pondering_rnn_test.runfiles/sonnet/sonnet/__init__.py", line 102, in <module>
from sonnet.python.ops.resampler import resampler
File "/home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/bazel-sandbox/9fc9d257-78be-40af-8d1a-aae9a9c04001-430/execroot/sonnet/bazel-out/local-fastbuild/bin/sonnet/python/pondering_rnn_test.runfiles/sonnet/sonnet/python/ops/resampler.py", line 33, in <module>
tf.resource_loader.get_path_to_datafile("_resampler.so"))
File "/home/kovalenko/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /home/kovalenko/.cache/bazel/_bazel_kovalenko/58aa37c963f86b8a87c9ab34d5bad7a3/bazel-sandbox/9fc9d257-78be-40af-8d1a-aae9a9c04001-430/execroot/sonnet/bazel-out/local-fastbuild/bin/sonnet/python/pondering_rnn_test.runfiles/sonnet/sonnet/python/ops/_resampler.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev
Use --strategy=TestRunner=standalone to disable sandboxing for the failing actions.
from sonnet.
Tutufa
commented on August 19, 2024
@malcolmreynolds do you build resampler with gcc 5., while tensorflow from pip is build by gcc 4.?
from sonnet.
Tutufa
commented on August 19, 2024
Yes, it was gcc version mismatch. Do the following:
- Compile r1.1 TF from github, install TF
- Compile Sonnet (switch to r1.1 in tensorflow dir inside sonnet), Install Sonnet
I used gcc 5.*
from sonnet.
Related Issues (20)
- how to remove model from gpu memory？ HOT 1
- Understanding how to use mixed precision training HOT 2
- Conv2D initialize error in tf function
- Typo HOT 1
- this portion of attention code looks incorrect
- Cannot install sonnet on mac m1 HOT 5
- Decaying learning rate HOT 1
- Can you explain the logic of updated_ema_cluster_size in VectorQuantizerEMA?
- Convert a Sonnet model to a Keras model HOT 1
- (Maybe) in-consistency between VQ-VAE paper and its implementation. HOT 2
- Regarding LearningRateSchedule for ADAM optimizer HOT 2
- Learning rate and optimizer used for the VQ-VAE-2 paper HOT 2
- I have a question about the perplexity term (in the VQ-VAE). HOT 2
- Issues using Sonnet HOT 3
- How to save the checkpoint for self-defined model?
- TensorArray Not Used on line 71 of sgd_test.py HOT 2
- Is there a stereotypical way to visualize the network strucuture when using Sonnet? HOT 1
- sonnet abstract module in tensorflow2 HOT 10
- Laplace smoothing for EMA codebook update
- Basic import fails on Colab: cannot import name 'context' from 'tensorflow.python' HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sonnet.