Comments (5)
kyzhouhzau
commented on August 17, 2024
it can automatically use GPU!
from bert-ner.
penut85420
commented on August 17, 2024
@yaoyao3 check pip list, make sure you only have tensorflow-gpu in it.
If you both have tensorflow & tensorflow-gpu, tf will choose cpu in default, at least in my case.
from bert-ner.
paulthemagno
commented on August 17, 2024
@kyzhouhzau I set --use_tpu=True but it doesn't see gpu
I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
INFO:tensorflow:Failed to find TPU: _TPUSystemMetadata(num_cores=0, num_hosts=0, num_of_cores_per_host=0, topology=None, devices=(_DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456, 13701800340323951977), _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 9332358750793188078)))
I1106 11:04:27.689568 140625411577664 tpu_system_metadata.py:156] Failed to find TPU: _TPUSystemMetadata(num_cores=0, num_hosts=0, num_of_cores_per_host=0, topology=None, devices=(_DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456, 13701800340323951977), _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 9332358750793188078)))
ERROR:tensorflow:Error recorded from training_loop: Cannot find any TPU cores in the system. Please double check Tensorflow master address and TPU worker(s). Available devices are (_DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456, 13701800340323951977), _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 9332358750793188078)).
E1106 11:04:27.690066 140625411577664 error_handling.py:70] Error recorded from training_loop: Cannot find any TPU cores in the system. Please double check Tensorflow master address and TPU worker(s). Available devices are (_DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456, 13701800340323951977), _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 9332358750793188078)).
INFO:tensorflow:training_loop marked as finished
I1106 11:04:27.690162 140625411577664 error_handling.py:96] training_loop marked as finished
WARNING:tensorflow:Reraising captured error
W1106 11:04:27.690225 140625411577664 error_handling.py:130] Reraising captured error
Traceback (most recent call last):
File "BERT_NER.py", line 697, in <module>
tf.app.run()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "BERT_NER.py", line 631, in main
estimator.train(input_fn=train_input_fn, max_steps=num_train_steps)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2876, in train
rendezvous.raise_errors()
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 131, in raise_errors
six.reraise(typ, value, traceback)
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2871, in train
saving_listeners=saving_listeners)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 364, in train
hooks.extend(self._convert_train_steps_to_hooks(steps, max_steps))
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2746, in _convert_train_steps_to_hooks
if ctx.is_running_on_cpu():
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_context.py", line 442, in is_running_on_cpu
self._validate_tpu_configuration()
File "/usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_context.py", line 613, in _validate_tpu_configuration
'are {}.'.format(tpu_system_metadata.devices))
RuntimeError: Cannot find any TPU cores in the system. Please double check Tensorflow master address and TPU worker(s). Available devices are (_DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 268435456, 13701800340323951977), _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 9332358750793188078)).
processed 40610 tokens with 4671 phrases; found: 4557 phrases; correct: 4104.from bert-ner.
penut85420
commented on August 17, 2024
@paulthemagno i think tpu is different from gpu, leave that option be false.
from bert-ner.
paulthemagno
commented on August 17, 2024
@kyzhouhzau I tried to set --use_tpu=False and it sees GPU, but only allocates a fixed quantity of memory on it (about 100Mib). Anyway all the computation is executed on CPU.
from bert-ner.
Related Issues (20)
- When I use my own test data which have ten million rows, but the model can only predict 4508 rows. I don't know what happend. There is no problem when I use original test data.
- [ERROR] TPUEstimator with BestExporter: ValueError: slice index 1 of dimension 0 out of bounds. HOT 1
- tensorflow version and how to Frozen bert? HOT 1
- eval_metric_ops[confusion_matrix] must be Operation or Tensor HOT 3
- How can i make a demo?
- In eval. Why take (num_labels - 1) in function streaming_confusion_matrix
- Reproduce CoNLL score
- Client for NER
- WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 25 vs previous value: 25. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize
- crf=False is coded as string therefore it does not work
- 为什么预测会出现BB、EE这种错误呢？
- What about license? HOT 3
- BERT_NER.py#L450 same code in if-else in both branches
- absl.flags._exceptions.UnparsedFlagAccessError: Trying to access flag --do_train before flags were parsed.
- 像您请教一下为什么训练的时候并没有出现loss以及epoch HOT 1
- Use GPU for inference
- 你好，BERT文件夹怎么是空的呢？ HOT 1
- A new public notebook NER git
- requirements.txt file?? HOT 1
- 训练报错
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bert-ner.