stonybrooknlp / deformer Goto Github PK

View Code? Open in Web Editor NEW

120.0 23.0 15.0 440 KB

[ACL 2020] DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering

License: MIT License

Python 98.64% Shell 1.36%

acl2020 deformer question-answering transformer decomposition

deformer's People

Contributors

Stargazers

Watchers

Forkers

yyht daishu7 luisfredgs anshiquanshu66 jamon96 zhangtongxuan woffee gdh756462786 phoebussi jxnevergiveup ben-wu14 zcf131016 varunsabnis yfeng-44 hippoley

deformer's Issues

运行bug

1.生成训练和评估数据提示：没有找到‘tokenizers’这个模块，具体可见‘features/feature_bert_classify.py’第一行。

2.该代码是在什么环境下训练的，TPU还是GPU？

Error running command from README for benchmarking inference latency

Hello,

I was able to follow the README to run the evaluation on the Bert and EBert models using the provided checkpoints. When I moved into the profiling section, I successfully ran the flop profiling as well as the inference latency benchmarking for the Ebert model. However, there was an error when I tried to run the given inference latency benchmarking for the provided Bert model.

I was able to see that the error came from line 33 in bert.py. The model specified in qa_bert.py does not provide the right shape for the "inputs" variable that is used to extract the "input_ids" and "token_type_ids" in this line. I was able to get past this error by changing this in qa_bert.py, creating another Tensor placeholder for segment_ids_ph (with the same dimensions as the placeholder for input_ids_ph) in the export_graph function (that was previously commented out, line 94), and then included both of these tensors so that the implicit function call to the "call" method of the model in bert.py would have the correct "inputs" variable.

I wanted to ensure that this would be a correct way to fix this issue, to check whether or not I am using the repository correctly. I have attached a picture of the updated code.

Thank you!

Questions about sbert fine-tuning and model deployment

When fine-tuning sbert, I saw the following command “python tools/explore_hp.py -p data/race-sbert-s9.json -n 50 -t race 2>&1 | tee data/race-sbert-explore-s9.log”, there is data set race-sbert-s9.json, can you upload this data set? It also includes the qqp task.
See that there is a serve.py file under the project, can you give an example of the use of this file?
Thank you very much!

When will source code be released?

Hello. When will source code be released? I'm very interested in this implementation.

What is bert_model.ckpt file?

Hello, when I ran eval.py, I got an error:

Failed to get matching files on ./data/ckpt/init/uncased_base/bert_model.ckpt

So, what is the bert_model.ckpt file, and can you tell me how to get it?
Thank you!

When will source code be released? I'm very interested in this implementation. I hope the source code canbe released as soon as possible?

Permission errors while downloading the checkpoints from google cloud

I was unable to download the checkpoints from google cloud SDK Shell. The bucket item list is protected and I don't have access the access to it. Without the checkpoints, I can't infer the models.

Any bug on RACE?

Hi Qingqing,

Thanks for releasing great code.

I have successfully trained sbert for SQuAD. But when I train original bert on RACE, the loss can not convergence even the number of training dataset is 2 (=1 is ok). Is there any bug about the RACE dataset?

If it is convenient, could you please provide me serveral sbert for RACE, such as sbert-s7/sbert-s9/sbert-s10?

Thanks, Looking forward to your reply!

Best,
Deming

please use tensorflow

could you realize it with tensorlfow?

Not found profile.py

Seems to be missing profile.py

error on training on GPU？

Hello,

I followed the instruction until training bert and ebert on qqp dataset. I trained them on GPU but did not get a reasonable result

I trained them with the commond:

python train.py -m bert -t qqp 2>&1 | tee data/qqp-bert-train.log
python train.py -m ebert -t qqp 2>&1 | tee data/qqp-ebert-train.log

and the train log file on bert are :

_WARNING:tensorflow:From /data1/hwt/deformer/common/optimizer.py:91: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

INFO:2020-12-11_18:12:51.913:/data1/hwt/deformer/common/config.py:130: config_file: /data1/hwt/deformer/config/bert_classifier.ini
INFO:2020-12-11_18:12:51.915:/data1/hwt/deformer/common/config.py:79: �[1m�[34mtask set to env qqp instead of provided �[0m
INFO:2020-12-11_18:12:51.916:/data1/hwt/deformer/common/config.py:79: �[1m�[34mmode set to env train instead of provided train�[0m
INFO:2020-12-11_18:12:51.917:/data1/hwt/deformer/common/config.py:96: (train) dataset_file: /data1/hwt/deformer/data/datasets/converted/bert/qqp-train.327464.tfrecord
WARNING:tensorflow:From train.py:18: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

INFO:2020-12-11_18:12:51.919:train.py:28: config:
attention_dropout_prob: 0.1
attention_head_size: 64
bfloat16:
checkpoint_dir: /data1/hwt/deformer/data/ckpt/bert-base/qqp
data_dir: /data1/hwt/deformer/data
dataset_file: /data1/hwt/deformer/data/datasets/converted/bert/qqp-train.327464.tfrecord
dataset_size: 327464
debug: False
dev_batch_size: 16
epochs: 3
ground_truth_file: /data1/hwt/deformer/data/datasets/converted/bert/qqp-dev.*.jsonl
hidden_dropout_prob: 0.1
hidden_size: 768
inference_graph: /data1/hwt/deformer/data/ckpt/bert/qqp_bert_infer.pb
init_checkpoint: /data1/hwt/deformer/data/ckpt/init/uncased_base/bert_model.ckpt
initializer_range: 0.02
input_buffer_size: 2000
input_num_threads: 8
intermediate_act_fn: gelu
intermediate_size: 3072
iterations_per_loop: 1000
keep_checkpoint_max: 20
learning_rate: 5e-05
lower_case: True
max_first_length: 40
max_position_embeddings: 512
max_seq_length: 100
mode: train
model: bert
num_choices: 0
num_classes: 2
num_heads: 12
num_hidden_layers: 12
num_tpu_cores: 8
num_train_steps: 30699
num_warmup_steps: 4604
optimize_padding: False
output_file: /data1/hwt/deformer/data/predictions/bert/qqp-dev-predictions.json
print_steps: 100
random_seed: 0
steps_per_checkpoint: 1000
task: qqp
tpu_name:
train_batch_size: 32
type_vocab_size: 2
use_host_call: True
use_replace_map: True
use_tpu: False
vocab_file: /data1/hwt/deformer/data/res/bert.vocab
vocab_size: 30522
warmup_ratio: 0.15
The current process just got forked. Disabling parallelism to avoid deadlocks...
To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /data1/hwt/deformer/common/tf_util.py:116: The name tf.keras.initializers.TruncatedNormal is deprecated. Please use tf.compat.v1.keras.initializers.TruncatedNormal instead.

WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/initializers.py:94: calling TruncatedNormal.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder..model_fn at 0x7f886427d268>) includes params argument, but params are not passed to Estimator.
WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.
INFO:2020-12-11_18:12:52.653:train.py:33: begin training for 30699 steps....
WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/data/util/random_seed.py:58: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
INFO:2020-12-11_18:12:53.027:/data1/hwt/deformer/common/builder.py:48: *** Features ***
INFO:2020-12-11_18:12:53.027:/data1/hwt/deformer/common/builder.py:50: name=feature_id, shape=(32,)
INFO:2020-12-11_18:12:53.027:/data1/hwt/deformer/common/builder.py:50: name=input_ids, shape=(32, 100)
INFO:2020-12-11_18:12:53.027:/data1/hwt/deformer/common/builder.py:50: name=segment_ids, shape=(32, 100)
WARNING:tensorflow:From /data1/hwt/deformer/common/builder.py:63: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From /data1/hwt/deformer/common/builder.py:107: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

INFO:2020-12-11_18:12:56.598:/data1/hwt/deformer/common/builder.py:109: **** Initialized Variables ****
INFO:2020-12-11_18:12:56.598:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/word_embeddings:0, shape=(30522, 768)
INFO:2020-12-11_18:12:56.598:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/token_type_embeddings:0, shape=(2, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/position_embeddings:0, shape=(512, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.599:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.600:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.601:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.602:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.603:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/pooler/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/pooler/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.604:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/classifier/dense/kernel:0, shape=(768, 2)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/classifier/dense/bias:0, shape=(2,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:117: **** Trainable Variables ****
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/embeddings/word_embeddings:0, shape=(30522, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/embeddings/token_type_embeddings:0, shape=(2, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/embeddings/position_embeddings:0, shape=(512, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/embeddings/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/embeddings/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_0/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.605:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_1/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_2/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.606:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_3/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_4/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.607:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_5/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_6/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.608:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_7/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.609:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_8/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_9/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.610:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_10/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/encoder/layer_11/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/pooler/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/bert/pooler/dense/bias:0, shape=(768,)
INFO:2020-12-11_18:12:56.611:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/classifier/dense/kernel:0, shape=(768, 2)
INFO:2020-12-11_18:12:56.612:/data1/hwt/deformer/common/builder.py:123: name=bert_classifier/classifier/dense/bias:0, shape=(2,)
WARNING:tensorflow:From /data1/hwt/deformer/common/optimizer.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.

WARNING:tensorflow:From /data1/hwt/deformer/common/optimizer.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

WARNING:tensorflow:From /data1/hwt/deformer/common/optimizer.py:133: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /data1/hwt/deformer/common/builder.py:195: The name tf.train.LoggingTensorHook is deprecated. Please use tf.estimator.LoggingTensorHook instead.

WARNING:tensorflow:From /data1/hwt/deformer/tasks/classifier.py:78: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.

WARNING:tensorflow:From /data1/hwt/deformer/tasks/classifier.py:87: The name tf.metrics.mean is deprecated. Please use tf.compat.v1.metrics.mean instead.

WARNING:tensorflow:From /data1/hwt/deformer/common/builder.py:199: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

2020-12-11 18:13:07.068641: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2020-12-11 18:13:07.103862: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2020-12-11 18:13:07.107824: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55b5889265d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-12-11 18:13:07.107875: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-12-11 18:13:07.112465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-12-11 18:13:07.369551: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55b5913c0bf0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-12-11 18:13:07.369605: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): TITAN RTX, Compute Capability 7.5
2020-12-11 18:13:07.369620: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (1): TITAN RTX, Compute Capability 7.5
2020-12-11 18:13:07.373568: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties:
name: TITAN RTX major: 7 minor: 5 memoryClockRate(GHz): 1.77
pciBusID: 0000:1a:00.0
2020-12-11 18:13:07.376250: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 1 with properties:
name: TITAN RTX major: 7 minor: 5 memoryClockRate(GHz): 1.77
pciBusID: 0000:89:00.0
2020-12-11 18:13:07.376647: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-12-11 18:13:07.378811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-12-11 18:13:07.380499: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-12-11 18:13:07.380788: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-12-11 18:13:07.382088: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-12-11 18:13:07.383096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-12-11 18:13:07.386400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-12-11 18:13:07.391995: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0, 1
2020-12-11 18:13:07.392043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-12-11 18:13:07.395613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-11 18:13:07.395629: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186] 0 1
2020-12-11 18:13:07.395636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0: N N
2020-12-11 18:13:07.395641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 1: N N
2020-12-11 18:13:07.400003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 22080 MB memory) -> physical GPU (device: 0, name: TITAN RTX, pci bus id: 0000:1a:00.0, compute capability: 7.5)
2020-12-11 18:13:07.401819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 16707 MB memory) -> physical GPU (device: 1, name: TITAN RTX, pci bus id: 0000:89:00.0, compute capability: 7.5)
2020-12-11 18:13:31.147009: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py:963: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
INFO:2020-12-11_20:33:44.190:train.py:38: training ended!
INFO:2020-12-11_20:33:44.191:train.py:39: all done, took 2:20:52.271457 s!_

and the eval log file on bert are :

_WARNING:tensorflow:From /data1/hwt/deformer/common/optimizer.py:91: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

INFO:2020-12-11_20:43:58.792:/data1/hwt/deformer/common/config.py:130: config_file: /data1/hwt/deformer/config/bert_classifier.ini
INFO:2020-12-11_20:43:58.793:/data1/hwt/deformer/common/config.py:79: �[1m�[34mtask set to env qqp instead of provided �[0m
INFO:2020-12-11_20:43:58.794:/data1/hwt/deformer/common/config.py:79: �[1m�[34mmode set to env dev instead of provided train�[0m
INFO:2020-12-11_20:43:58.795:/data1/hwt/deformer/common/config.py:96: (dev) dataset_file: /data1/hwt/deformer/data/datasets/converted/bert/qqp-dev.40430.tfrecord
WARNING:tensorflow:From eval.py:24: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

INFO:2020-12-11_20:43:58.797:eval.py:31: config:
attention_dropout_prob: 0.1
attention_head_size: 64
bfloat16:
checkpoint_dir: /data1/hwt/deformer/data/ckpt/bert-base/qqp
checkpoint_path: None
data_dir: /data1/hwt/deformer/data
dataset_file: /data1/hwt/deformer/data/datasets/converted/bert/qqp-dev.40430.tfrecord
dataset_size: 40430
debug: False
dev_batch_size: 16
epochs: 3
ground_truth_file: /data1/hwt/deformer/data/datasets/converted/bert/qqp-dev.40430.jsonl
hidden_dropout_prob: 0.1
hidden_size: 768
inference_graph: /data1/hwt/deformer/data/ckpt/bert/qqp_bert_infer.pb
init_checkpoint: /data1/hwt/deformer/data/ckpt/init/uncased_base/bert_model.ckpt
initializer_range: 0.02
input_buffer_size: 2000
input_num_threads: 8
intermediate_act_fn: gelu
intermediate_size: 3072
iterate_checkpoints: False
iterate_timeout: 3600
iterations_per_loop: 1000
keep_checkpoint_max: 20
learning_rate: 5e-05
lower_case: True
max_first_length: 40
max_position_embeddings: 512
max_seq_length: 100
mode: dev
model: bert
num_choices: 0
num_classes: 2
num_heads: 12
num_hidden_layers: 12
num_tpu_cores: 8
optimize_padding: False
output_file: /data1/hwt/deformer/data/predictions/bert/qqp-dev-predictions.json
print_steps: 100
random_seed: 0
steps_per_checkpoint: 1000
task: qqp
tpu_name:
train_batch_size: 32
type_vocab_size: 2
use_host_call: True
use_replace_map: True
use_tpu: False
vocab_file: /data1/hwt/deformer/data/res/bert.vocab
vocab_size: 30522
warmup_ratio: 0.15
The current process just got forked. Disabling parallelism to avoid deadlocks...
To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /data1/hwt/deformer/common/tf_util.py:116: The name tf.keras.initializers.TruncatedNormal is deprecated. Please use tf.compat.v1.keras.initializers.TruncatedNormal instead.

WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/initializers.py:94: calling TruncatedNormal.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder..model_fn at 0x7f7c59839268>) includes params argument, but params are not passed to Estimator.
WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.
INFO:2020-12-11_20:43:59.573:eval.py:42: loading examples from /data1/hwt/deformer/data/datasets/converted/bert/qqp-dev.40430.jsonl....
INFO:2020-12-11_20:44:01.166:eval.py:48: begin evaluating /data1/hwt/deformer/data/ckpt/bert-base/qqp/model.ckpt-30699...
WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:2020-12-11_20:44:01.598:/data1/hwt/deformer/common/builder.py:48: *** Features ***
INFO:2020-12-11_20:44:01.598:/data1/hwt/deformer/common/builder.py:50: name=feature_id, shape=(?,)
INFO:2020-12-11_20:44:01.598:/data1/hwt/deformer/common/builder.py:50: name=input_ids, shape=(?, 100)
INFO:2020-12-11_20:44:01.598:/data1/hwt/deformer/common/builder.py:50: name=segment_ids, shape=(?, 100)
WARNING:tensorflow:From /data1/hwt/deformer/common/builder.py:63: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From /data1/hwt/deformer/common/builder.py:107: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:109: **** Initialized Variables ****
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/word_embeddings:0, shape=(30522, 768)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/token_type_embeddings:0, shape=(2, 768)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/position_embeddings:0, shape=(512, 768)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/embeddings/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.059:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_0/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.060:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_1/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.061:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_2/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_3/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.062:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_4/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.063:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_5/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.064:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_6/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_7/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.065:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_8/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.066:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_9/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_10/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.067:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/query/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/query/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/key/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/key/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/value/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/self/value/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/attention/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/intermediate/dense/kernel:0, shape=(768, 3072)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/intermediate/dense/bias:0, shape=(3072,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/dense/kernel:0, shape=(3072, 768)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/layer_norm/gamma:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/encoder/layer_11/output/layer_norm/beta:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/pooler/dense/kernel:0, shape=(768, 768)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/bert/pooler/dense/bias:0, shape=(768,)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/classifier/dense/kernel:0, shape=(768, 2)
INFO:2020-12-11_20:44:06.068:/data1/hwt/deformer/common/builder.py:114: name=bert_classifier/classifier/dense/bias:0, shape=(2,)
WARNING:tensorflow:From /home/hwt/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
2020-12-11 20:44:06.420797: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2020-12-11 20:44:06.456048: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2020-12-11 20:44:06.460253: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55aeaaefa890 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-12-11 20:44:06.460297: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-12-11 20:44:06.465020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-12-11 20:44:06.747001: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55aeaade4fd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-12-11 20:44:06.747060: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): TITAN RTX, Compute Capability 7.5
2020-12-11 20:44:06.747075: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (1): TITAN RTX, Compute Capability 7.5
2020-12-11 20:44:06.750935: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties:
name: TITAN RTX major: 7 minor: 5 memoryClockRate(GHz): 1.77
pciBusID: 0000:1a:00.0
2020-12-11 20:44:06.751883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 1 with properties:
name: TITAN RTX major: 7 minor: 5 memoryClockRate(GHz): 1.77
pciBusID: 0000:89:00.0
2020-12-11 20:44:06.752308: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-12-11 20:44:06.754967: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2020-12-11 20:44:06.757226: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2020-12-11 20:44:06.757793: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2020-12-11 20:44:06.760374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2020-12-11 20:44:06.761648: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2020-12-11 20:44:06.765756: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-12-11 20:44:06.769820: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0, 1
2020-12-11 20:44:06.769868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-12-11 20:44:06.772734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-11 20:44:06.772752: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186] 0 1
2020-12-11 20:44:06.772758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0: N N
2020-12-11 20:44:06.772763: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 1: N N
2020-12-11 20:44:06.776361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 22080 MB memory) -> physical GPU (device: 0, name: TITAN RTX, pci bus id: 0000:1a:00.0, compute capability: 7.5)
2020-12-11 20:44:06.777403: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 770 MB memory) -> physical GPU (device: 1, name: TITAN RTX, pci bus id: 0000:89:00.0, compute capability: 7.5)
2020-12-11 20:44:09.491162: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
INFO:2020-12-11_20:44:10.350:eval.py:120: model.ckpt-30699, predicted 10/(2526) batches
INFO:2020-12-11_20:44:10.925:eval.py:120: model.ckpt-30699, predicted 20/(2526) batches
INFO:2020-12-11_20:44:11.521:eval.py:120: model.ckpt-30699, predicted 30/(2526) batches
INFO:2020-12-11_20:44:12.126:eval.py:120: model.ckpt-30699, predicted 40/(2526) batches
INFO:2020-12-11_20:44:12.719:eval.py:120: model.ckpt-30699, predicted 50/(2526) batches
INFO:2020-12-11_20:44:13.327:eval.py:120: model.ckpt-30699, predicted 60/(2526) batches
INFO:2020-12-11_20:44:13.929:eval.py:120: model.ckpt-30699, predicted 70/(2526) batches
INFO:2020-12-11_20:44:14.522:eval.py:120: model.ckpt-30699, predicted 80/(2526) batches
INFO:2020-12-11_20:44:15.108:eval.py:120: model.ckpt-30699, predicted 90/(2526) batches
INFO:2020-12-11_20:44:15.726:eval.py:120: model.ckpt-30699, predicted 100/(2526) batches
INFO:2020-12-11_20:44:16.330:eval.py:120: model.ckpt-30699, predicted 110/(2526) batches
INFO:2020-12-11_20:44:16.928:eval.py:120: model.ckpt-30699, predicted 120/(2526) batches
INFO:2020-12-11_20:44:17.530:eval.py:120: model.ckpt-30699, predicted 130/(2526) batches
INFO:2020-12-11_20:44:18.134:eval.py:120: model.ckpt-30699, predicted 140/(2526) batches
INFO:2020-12-11_20:44:18.719:eval.py:120: model.ckpt-30699, predicted 150/(2526) batches
INFO:2020-12-11_20:44:19.299:eval.py:120: model.ckpt-30699, predicted 160/(2526) batches
INFO:2020-12-11_20:44:19.889:eval.py:120: model.ckpt-30699, predicted 170/(2526) batches
INFO:2020-12-11_20:44:20.493:eval.py:120: model.ckpt-30699, predicted 180/(2526) batches
INFO:2020-12-11_20:44:21.092:eval.py:120: model.ckpt-30699, predicted 190/(2526) batches
INFO:2020-12-11_20:44:21.688:eval.py:120: model.ckpt-30699, predicted 200/(2526) batches
INFO:2020-12-11_20:44:22.284:eval.py:120: model.ckpt-30699, predicted 210/(2526) batches
INFO:2020-12-11_20:44:22.882:eval.py:120: model.ckpt-30699, predicted 220/(2526) batches
INFO:2020-12-11_20:44:23.475:eval.py:120: model.ckpt-30699, predicted 230/(2526) batches
INFO:2020-12-11_20:44:24.088:eval.py:120: model.ckpt-30699, predicted 240/(2526) batches
INFO:2020-12-11_20:44:24.691:eval.py:120: model.ckpt-30699, predicted 250/(2526) batches
INFO:2020-12-11_20:44:25.302:eval.py:120: model.ckpt-30699, predicted 260/(2526) batches
INFO:2020-12-11_20:44:25.918:eval.py:120: model.ckpt-30699, predicted 270/(2526) batches
INFO:2020-12-11_20:44:26.538:eval.py:120: model.ckpt-30699, predicted 280/(2526) batches
INFO:2020-12-11_20:44:27.131:eval.py:120: model.ckpt-30699, predicted 290/(2526) batches
INFO:2020-12-11_20:44:27.741:eval.py:120: model.ckpt-30699, predicted 300/(2526) batches
INFO:2020-12-11_20:44:28.351:eval.py:120: model.ckpt-30699, predicted 310/(2526) batches
INFO:2020-12-11_20:44:28.955:eval.py:120: model.ckpt-30699, predicted 320/(2526) batches
INFO:2020-12-11_20:44:29.551:eval.py:120: model.ckpt-30699, predicted 330/(2526) batches
INFO:2020-12-11_20:44:30.140:eval.py:120: model.ckpt-30699, predicted 340/(2526) batches
INFO:2020-12-11_20:44:30.726:eval.py:120: model.ckpt-30699, predicted 350/(2526) batches
INFO:2020-12-11_20:44:31.306:eval.py:120: model.ckpt-30699, predicted 360/(2526) batches
INFO:2020-12-11_20:44:31.890:eval.py:120: model.ckpt-30699, predicted 370/(2526) batches
INFO:2020-12-11_20:44:32.479:eval.py:120: model.ckpt-30699, predicted 380/(2526) batches
INFO:2020-12-11_20:44:33.098:eval.py:120: model.ckpt-30699, predicted 390/(2526) batches
INFO:2020-12-11_20:44:33.694:eval.py:120: model.ckpt-30699, predicted 400/(2526) batches
INFO:2020-12-11_20:44:34.298:eval.py:120: model.ckpt-30699, predicted 410/(2526) batches
INFO:2020-12-11_20:44:34.890:eval.py:120: model.ckpt-30699, predicted 420/(2526) batches
INFO:2020-12-11_20:44:35.494:eval.py:120: model.ckpt-30699, predicted 430/(2526) batches
INFO:2020-12-11_20:44:36.100:eval.py:120: model.ckpt-30699, predicted 440/(2526) batches
INFO:2020-12-11_20:44:36.703:eval.py:120: model.ckpt-30699, predicted 450/(2526) batches
INFO:2020-12-11_20:44:37.313:eval.py:120: model.ckpt-30699, predicted 460/(2526) batches
INFO:2020-12-11_20:44:37.914:eval.py:120: model.ckpt-30699, predicted 470/(2526) batches
INFO:2020-12-11_20:44:38.511:eval.py:120: model.ckpt-30699, predicted 480/(2526) batches
INFO:2020-12-11_20:44:39.094:eval.py:120: model.ckpt-30699, predicted 490/(2526) batches
INFO:2020-12-11_20:44:39.708:eval.py:120: model.ckpt-30699, predicted 500/(2526) batches
INFO:2020-12-11_20:44:40.341:eval.py:120: model.ckpt-30699, predicted 510/(2526) batches
INFO:2020-12-11_20:44:40.957:eval.py:120: model.ckpt-30699, predicted 520/(2526) batches
INFO:2020-12-11_20:44:41.545:eval.py:120: model.ckpt-30699, predicted 530/(2526) batches
INFO:2020-12-11_20:44:42.157:eval.py:120: model.ckpt-30699, predicted 540/(2526) batches
INFO:2020-12-11_20:44:42.758:eval.py:120: model.ckpt-30699, predicted 550/(2526) batches
INFO:2020-12-11_20:44:43.360:eval.py:120: model.ckpt-30699, predicted 560/(2526) batches
INFO:2020-12-11_20:44:43.977:eval.py:120: model.ckpt-30699, predicted 570/(2526) batches
INFO:2020-12-11_20:44:44.570:eval.py:120: model.ckpt-30699, predicted 580/(2526) batches
INFO:2020-12-11_20:44:45.174:eval.py:120: model.ckpt-30699, predicted 590/(2526) batches
INFO:2020-12-11_20:44:45.778:eval.py:120: model.ckpt-30699, predicted 600/(2526) batches
INFO:2020-12-11_20:44:46.385:eval.py:120: model.ckpt-30699, predicted 610/(2526) batches
INFO:2020-12-11_20:44:46.971:eval.py:120: model.ckpt-30699, predicted 620/(2526) batches
INFO:2020-12-11_20:44:47.580:eval.py:120: model.ckpt-30699, predicted 630/(2526) batches
INFO:2020-12-11_20:44:48.172:eval.py:120: model.ckpt-30699, predicted 640/(2526) batches
INFO:2020-12-11_20:44:48.790:eval.py:120: model.ckpt-30699, predicted 650/(2526) batches
INFO:2020-12-11_20:44:49.401:eval.py:120: model.ckpt-30699, predicted 660/(2526) batches
INFO:2020-12-11_20:44:50.017:eval.py:120: model.ckpt-30699, predicted 670/(2526) batches
INFO:2020-12-11_20:44:50.619:eval.py:120: model.ckpt-30699, predicted 680/(2526) batches
INFO:2020-12-11_20:44:51.211:eval.py:120: model.ckpt-30699, predicted 690/(2526) batches
INFO:2020-12-11_20:44:51.817:eval.py:120: model.ckpt-30699, predicted 700/(2526) batches
INFO:2020-12-11_20:44:52.435:eval.py:120: model.ckpt-30699, predicted 710/(2526) batches
INFO:2020-12-11_20:44:53.038:eval.py:120: model.ckpt-30699, predicted 720/(2526) batches
INFO:2020-12-11_20:44:53.642:eval.py:120: model.ckpt-30699, predicted 730/(2526) batches
INFO:2020-12-11_20:44:54.238:eval.py:120: model.ckpt-30699, predicted 740/(2526) batches
INFO:2020-12-11_20:44:54.830:eval.py:120: model.ckpt-30699, predicted 750/(2526) batches
INFO:2020-12-11_20:44:55.416:eval.py:120: model.ckpt-30699, predicted 760/(2526) batches
INFO:2020-12-11_20:44:56.007:eval.py:120: model.ckpt-30699, predicted 770/(2526) batches
INFO:2020-12-11_20:44:56.604:eval.py:120: model.ckpt-30699, predicted 780/(2526) batches
INFO:2020-12-11_20:44:57.218:eval.py:120: model.ckpt-30699, predicted 790/(2526) batches
INFO:2020-12-11_20:44:57.843:eval.py:120: model.ckpt-30699, predicted 800/(2526) batches
INFO:2020-12-11_20:44:58.454:eval.py:120: model.ckpt-30699, predicted 810/(2526) batches
INFO:2020-12-11_20:44:59.046:eval.py:120: model.ckpt-30699, predicted 820/(2526) batches
INFO:2020-12-11_20:44:59.658:eval.py:120: model.ckpt-30699, predicted 830/(2526) batches
INFO:2020-12-11_20:45:00.265:eval.py:120: model.ckpt-30699, predicted 840/(2526) batches
INFO:2020-12-11_20:45:00.885:eval.py:120: model.ckpt-30699, predicted 850/(2526) batches
INFO:2020-12-11_20:45:01.504:eval.py:120: model.ckpt-30699, predicted 860/(2526) batches
INFO:2020-12-11_20:45:02.110:eval.py:120: model.ckpt-30699, predicted 870/(2526) batches
INFO:2020-12-11_20:45:02.698:eval.py:120: model.ckpt-30699, predicted 880/(2526) batches
INFO:2020-12-11_20:45:03.296:eval.py:120: model.ckpt-30699, predicted 890/(2526) batches
INFO:2020-12-11_20:45:03.904:eval.py:120: model.ckpt-30699, predicted 900/(2526) batches
INFO:2020-12-11_20:45:04.503:eval.py:120: model.ckpt-30699, predicted 910/(2526) batches
INFO:2020-12-11_20:45:05.130:eval.py:120: model.ckpt-30699, predicted 920/(2526) batches
INFO:2020-12-11_20:45:05.749:eval.py:120: model.ckpt-30699, predicted 930/(2526) batches
INFO:2020-12-11_20:45:06.359:eval.py:120: model.ckpt-30699, predicted 940/(2526) batches
INFO:2020-12-11_20:45:06.956:eval.py:120: model.ckpt-30699, predicted 950/(2526) batches
INFO:2020-12-11_20:45:07.560:eval.py:120: model.ckpt-30699, predicted 960/(2526) batches
INFO:2020-12-11_20:45:08.182:eval.py:120: model.ckpt-30699, predicted 970/(2526) batches
INFO:2020-12-11_20:45:08.793:eval.py:120: model.ckpt-30699, predicted 980/(2526) batches
INFO:2020-12-11_20:45:09.395:eval.py:120: model.ckpt-30699, predicted 990/(2526) batches
INFO:2020-12-11_20:45:09.973:eval.py:120: model.ckpt-30699, predicted 1000/(2526) batches
INFO:2020-12-11_20:45:10.566:eval.py:120: model.ckpt-30699, predicted 1010/(2526) batches
INFO:2020-12-11_20:45:11.157:eval.py:120: model.ckpt-30699, predicted 1020/(2526) batches
INFO:2020-12-11_20:45:11.762:eval.py:120: model.ckpt-30699, predicted 1030/(2526) batches
INFO:2020-12-11_20:45:12.364:eval.py:120: model.ckpt-30699, predicted 1040/(2526) batches
INFO:2020-12-11_20:45:12.974:eval.py:120: model.ckpt-30699, predicted 1050/(2526) batches
INFO:2020-12-11_20:45:13.591:eval.py:120: model.ckpt-30699, predicted 1060/(2526) batches
INFO:2020-12-11_20:45:14.194:eval.py:120: model.ckpt-30699, predicted 1070/(2526) batches
INFO:2020-12-11_20:45:14.788:eval.py:120: model.ckpt-30699, predicted 1080/(2526) batches
INFO:2020-12-11_20:45:15.386:eval.py:120: model.ckpt-30699, predicted 1090/(2526) batches
INFO:2020-12-11_20:45:15.989:eval.py:120: model.ckpt-30699, predicted 1100/(2526) batches
INFO:2020-12-11_20:45:16.602:eval.py:120: model.ckpt-30699, predicted 1110/(2526) batches
INFO:2020-12-11_20:45:17.214:eval.py:120: model.ckpt-30699, predicted 1120/(2526) batches
INFO:2020-12-11_20:45:17.825:eval.py:120: model.ckpt-30699, predicted 1130/(2526) batches
INFO:2020-12-11_20:45:18.422:eval.py:120: model.ckpt-30699, predicted 1140/(2526) batches
INFO:2020-12-11_20:45:19.006:eval.py:120: model.ckpt-30699, predicted 1150/(2526) batches
INFO:2020-12-11_20:45:19.609:eval.py:120: model.ckpt-30699, predicted 1160/(2526) batches
INFO:2020-12-11_20:45:20.202:eval.py:120: model.ckpt-30699, predicted 1170/(2526) batches
INFO:2020-12-11_20:45:20.800:eval.py:120: model.ckpt-30699, predicted 1180/(2526) batches
INFO:2020-12-11_20:45:21.411:eval.py:120: model.ckpt-30699, predicted 1190/(2526) batches
INFO:2020-12-11_20:45:22.026:eval.py:120: model.ckpt-30699, predicted 1200/(2526) batches
INFO:2020-12-11_20:45:22.627:eval.py:120: model.ckpt-30699, predicted 1210/(2526) batches
INFO:2020-12-11_20:45:23.226:eval.py:120: model.ckpt-30699, predicted 1220/(2526) batches
INFO:2020-12-11_20:45:23.831:eval.py:120: model.ckpt-30699, predicted 1230/(2526) batches
INFO:2020-12-11_20:45:24.442:eval.py:120: model.ckpt-30699, predicted 1240/(2526) batches
INFO:2020-12-11_20:45:25.055:eval.py:120: model.ckpt-30699, predicted 1250/(2526) batches
INFO:2020-12-11_20:45:25.677:eval.py:120: model.ckpt-30699, predicted 1260/(2526) batches
INFO:2020-12-11_20:45:26.300:eval.py:120: model.ckpt-30699, predicted 1270/(2526) batches
INFO:2020-12-11_20:45:26.893:eval.py:120: model.ckpt-30699, predicted 1280/(2526) batches
INFO:2020-12-11_20:45:27.496:eval.py:120: model.ckpt-30699, predicted 1290/(2526) batches
INFO:2020-12-11_20:45:28.092:eval.py:120: model.ckpt-30699, predicted 1300/(2526) batches
INFO:2020-12-11_20:45:28.701:eval.py:120: model.ckpt-30699, predicted 1310/(2526) batches
INFO:2020-12-11_20:45:29.297:eval.py:120: model.ckpt-30699, predicted 1320/(2526) batches
INFO:2020-12-11_20:45:29.877:eval.py:120: model.ckpt-30699, predicted 1330/(2526) batches
INFO:2020-12-11_20:45:30.483:eval.py:120: model.ckpt-30699, predicted 1340/(2526) batches
INFO:2020-12-11_20:45:31.078:eval.py:120: model.ckpt-30699, predicted 1350/(2526) batches
INFO:2020-12-11_20:45:31.683:eval.py:120: model.ckpt-30699, predicted 1360/(2526) batches
INFO:2020-12-11_20:45:32.278:eval.py:120: model.ckpt-30699, predicted 1370/(2526) batches
INFO:2020-12-11_20:45:32.892:eval.py:120: model.ckpt-30699, predicted 1380/(2526) batches
INFO:2020-12-11_20:45:33.490:eval.py:120: model.ckpt-30699, predicted 1390/(2526) batches
INFO:2020-12-11_20:45:34.090:eval.py:120: model.ckpt-30699, predicted 1400/(2526) batches
INFO:2020-12-11_20:45:34.699:eval.py:120: model.ckpt-30699, predicted 1410/(2526) batches
INFO:2020-12-11_20:45:35.294:eval.py:120: model.ckpt-30699, predicted 1420/(2526) batches
INFO:2020-12-11_20:45:35.903:eval.py:120: model.ckpt-30699, predicted 1430/(2526) batches
INFO:2020-12-11_20:45:36.526:eval.py:120: model.ckpt-30699, predicted 1440/(2526) batches
INFO:2020-12-11_20:45:37.107:eval.py:120: model.ckpt-30699, predicted 1450/(2526) batches
INFO:2020-12-11_20:45:37.692:eval.py:120: model.ckpt-30699, predicted 1460/(2526) batches
INFO:2020-12-11_20:45:38.296:eval.py:120: model.ckpt-30699, predicted 1470/(2526) batches
INFO:2020-12-11_20:45:38.881:eval.py:120: model.ckpt-30699, predicted 1480/(2526) batches
INFO:2020-12-11_20:45:39.479:eval.py:120: model.ckpt-30699, predicted 1490/(2526) batches
INFO:2020-12-11_20:45:40.063:eval.py:120: model.ckpt-30699, predicted 1500/(2526) batches
INFO:2020-12-11_20:45:40.661:eval.py:120: model.ckpt-30699, predicted 1510/(2526) batches
INFO:2020-12-11_20:45:41.274:eval.py:120: model.ckpt-30699, predicted 1520/(2526) batches
INFO:2020-12-11_20:45:41.879:eval.py:120: model.ckpt-30699, predicted 1530/(2526) batches
INFO:2020-12-11_20:45:42.492:eval.py:120: model.ckpt-30699, predicted 1540/(2526) batches
INFO:2020-12-11_20:45:43.084:eval.py:120: model.ckpt-30699, predicted 1550/(2526) batches
INFO:2020-12-11_20:45:43.692:eval.py:120: model.ckpt-30699, predicted 1560/(2526) batches
INFO:2020-12-11_20:45:44.296:eval.py:120: model.ckpt-30699, predicted 1570/(2526) batches
INFO:2020-12-11_20:45:44.889:eval.py:120: model.ckpt-30699, predicted 1580/(2526) batches
INFO:2020-12-11_20:45:45.502:eval.py:120: model.ckpt-30699, predicted 1590/(2526) batches
INFO:2020-12-11_20:45:46.092:eval.py:120: model.ckpt-30699, predicted 1600/(2526) batches
INFO:2020-12-11_20:45:46.689:eval.py:120: model.ckpt-30699, predicted 1610/(2526) batches
INFO:2020-12-11_20:45:47.295:eval.py:120: model.ckpt-30699, predicted 1620/(2526) batches
INFO:2020-12-11_20:45:47.902:eval.py:120: model.ckpt-30699, predicted 1630/(2526) batches
INFO:2020-12-11_20:45:48.513:eval.py:120: model.ckpt-30699, predicted 1640/(2526) batches
INFO:2020-12-11_20:45:49.102:eval.py:120: model.ckpt-30699, predicted 1650/(2526) batches
INFO:2020-12-11_20:45:49.709:eval.py:120: model.ckpt-30699, predicted 1660/(2526) batches
INFO:2020-12-11_20:45:50.308:eval.py:120: model.ckpt-30699, predicted 1670/(2526) batches
INFO:2020-12-11_20:45:50.894:eval.py:120: model.ckpt-30699, predicted 1680/(2526) batches
INFO:2020-12-11_20:45:51.498:eval.py:120: model.ckpt-30699, predicted 1690/(2526) batches
INFO:2020-12-11_20:45:52.112:eval.py:120: model.ckpt-30699, predicted 1700/(2526) batches
INFO:2020-12-11_20:45:52.726:eval.py:120: model.ckpt-30699, predicted 1710/(2526) batches
INFO:2020-12-11_20:45:53.321:eval.py:120: model.ckpt-30699, predicted 1720/(2526) batches
INFO:2020-12-11_20:45:53.919:eval.py:120: model.ckpt-30699, predicted 1730/(2526) batches
INFO:2020-12-11_20:45:54.522:eval.py:120: model.ckpt-30699, predicted 1740/(2526) batches
INFO:2020-12-11_20:45:55.102:eval.py:120: model.ckpt-30699, predicted 1750/(2526) batches
INFO:2020-12-11_20:45:55.704:eval.py:120: model.ckpt-30699, predicted 1760/(2526) batches
INFO:2020-12-11_20:45:56.317:eval.py:120: model.ckpt-30699, predicted 1770/(2526) batches
INFO:2020-12-11_20:45:56.928:eval.py:120: model.ckpt-30699, predicted 1780/(2526) batches
INFO:2020-12-11_20:45:57.529:eval.py:120: model.ckpt-30699, predicted 1790/(2526) batches
INFO:2020-12-11_20:45:58.132:eval.py:120: model.ckpt-30699, predicted 1800/(2526) batches
INFO:2020-12-11_20:45:58.727:eval.py:120: model.ckpt-30699, predicted 1810/(2526) batches
INFO:2020-12-11_20:45:59.324:eval.py:120: model.ckpt-30699, predicted 1820/(2526) batches
INFO:2020-12-11_20:45:59.916:eval.py:120: model.ckpt-30699, predicted 1830/(2526) batches
INFO:2020-12-11_20:46:00.508:eval.py:120: model.ckpt-30699, predicted 1840/(2526) batches
INFO:2020-12-11_20:46:01.106:eval.py:120: model.ckpt-30699, predicted 1850/(2526) batches
INFO:2020-12-11_20:46:01.719:eval.py:120: model.ckpt-30699, predicted 1860/(2526) batches
INFO:2020-12-11_20:46:02.334:eval.py:120: model.ckpt-30699, predicted 1870/(2526) batches
INFO:2020-12-11_20:46:02.926:eval.py:120: model.ckpt-30699, predicted 1880/(2526) batches
INFO:2020-12-11_20:46:03.527:eval.py:120: model.ckpt-30699, predicted 1890/(2526) batches
INFO:2020-12-11_20:46:04.133:eval.py:120: model.ckpt-30699, predicted 1900/(2526) batches
INFO:2020-12-11_20:46:04.753:eval.py:120: model.ckpt-30699, predicted 1910/(2526) batches
INFO:2020-12-11_20:46:05.373:eval.py:120: model.ckpt-30699, predicted 1920/(2526) batches
INFO:2020-12-11_20:46:05.985:eval.py:120: model.ckpt-30699, predicted 1930/(2526) batches
INFO:2020-12-11_20:46:06.582:eval.py:120: model.ckpt-30699, predicted 1940/(2526) batches
INFO:2020-12-11_20:46:07.167:eval.py:120: model.ckpt-30699, predicted 1950/(2526) batches
INFO:2020-12-11_20:46:07.775:eval.py:120: model.ckpt-30699, predicted 1960/(2526) batches
INFO:2020-12-11_20:46:08.378:eval.py:120: model.ckpt-30699, predicted 1970/(2526) batches
INFO:2020-12-11_20:46:08.980:eval.py:120: model.ckpt-30699, predicted 1980/(2526) batches
INFO:2020-12-11_20:46:09.579:eval.py:120: model.ckpt-30699, predicted 1990/(2526) batches
INFO:2020-12-11_20:46:10.189:eval.py:120: model.ckpt-30699, predicted 2000/(2526) batches
INFO:2020-12-11_20:46:10.781:eval.py:120: model.ckpt-30699, predicted 2010/(2526) batches
INFO:2020-12-11_20:46:11.374:eval.py:120: model.ckpt-30699, predicted 2020/(2526) batches
INFO:2020-12-11_20:46:11.977:eval.py:120: model.ckpt-30699, predicted 2030/(2526) batches
INFO:2020-12-11_20:46:12.575:eval.py:120: model.ckpt-30699, predicted 2040/(2526) batches
INFO:2020-12-11_20:46:13.183:eval.py:120: model.ckpt-30699, predicted 2050/(2526) batches
INFO:2020-12-11_20:46:13.785:eval.py:120: model.ckpt-30699, predicted 2060/(2526) batches
INFO:2020-12-11_20:46:14.392:eval.py:120: model.ckpt-30699, predicted 2070/(2526) batches
INFO:2020-12-11_20:46:14.982:eval.py:120: model.ckpt-30699, predicted 2080/(2526) batches
INFO:2020-12-11_20:46:15.576:eval.py:120: model.ckpt-30699, predicted 2090/(2526) batches
INFO:2020-12-11_20:46:16.170:eval.py:120: model.ckpt-30699, predicted 2100/(2526) batches
INFO:2020-12-11_20:46:16.779:eval.py:120: model.ckpt-30699, predicted 2110/(2526) batches
INFO:2020-12-11_20:46:17.387:eval.py:120: model.ckpt-30699, predicted 2120/(2526) batches
INFO:2020-12-11_20:46:17.990:eval.py:120: model.ckpt-30699, predicted 2130/(2526) batches
INFO:2020-12-11_20:46:18.580:eval.py:120: model.ckpt-30699, predicted 2140/(2526) batches
INFO:2020-12-11_20:46:19.163:eval.py:120: model.ckpt-30699, predicted 2150/(2526) batches
INFO:2020-12-11_20:46:19.768:eval.py:120: model.ckpt-30699, predicted 2160/(2526) batches
INFO:2020-12-11_20:46:20.362:eval.py:120: model.ckpt-30699, predicted 2170/(2526) batches
INFO:2020-12-11_20:46:20.967:eval.py:120: model.ckpt-30699, predicted 2180/(2526) batches
INFO:2020-12-11_20:46:21.576:eval.py:120: model.ckpt-30699, predicted 2190/(2526) batches
INFO:2020-12-11_20:46:22.174:eval.py:120: model.ckpt-30699, predicted 2200/(2526) batches
INFO:2020-12-11_20:46:22.770:eval.py:120: model.ckpt-30699, predicted 2210/(2526) batches
INFO:2020-12-11_20:46:23.367:eval.py:120: model.ckpt-30699, predicted 2220/(2526) batches
INFO:2020-12-11_20:46:23.978:eval.py:120: model.ckpt-30699, predicted 2230/(2526) batches
INFO:2020-12-11_20:46:24.589:eval.py:120: model.ckpt-30699, predicted 2240/(2526) batches
INFO:2020-12-11_20:46:25.195:eval.py:120: model.ckpt-30699, predicted 2250/(2526) batches
INFO:2020-12-11_20:46:25.803:eval.py:120: model.ckpt-30699, predicted 2260/(2526) batches
INFO:2020-12-11_20:46:26.399:eval.py:120: model.ckpt-30699, predicted 2270/(2526) batches
INFO:2020-12-11_20:46:26.989:eval.py:120: model.ckpt-30699, predicted 2280/(2526) batches
INFO:2020-12-11_20:46:27.601:eval.py:120: model.ckpt-30699, predicted 2290/(2526) batches
INFO:2020-12-11_20:46:28.199:eval.py:120: model.ckpt-30699, predicted 2300/(2526) batches
INFO:2020-12-11_20:46:28.787:eval.py:120: model.ckpt-30699, predicted 2310/(2526) batches
INFO:2020-12-11_20:46:29.396:eval.py:120: model.ckpt-30699, predicted 2320/(2526) batches
INFO:2020-12-11_20:46:30.004:eval.py:120: model.ckpt-30699, predicted 2330/(2526) batches
INFO:2020-12-11_20:46:30.597:eval.py:120: model.ckpt-30699, predicted 2340/(2526) batches
INFO:2020-12-11_20:46:31.176:eval.py:120: model.ckpt-30699, predicted 2350/(2526) batches
INFO:2020-12-11_20:46:31.774:eval.py:120: model.ckpt-30699, predicted 2360/(2526) batches
INFO:2020-12-11_20:46:32.393:eval.py:120: model.ckpt-30699, predicted 2370/(2526) batches
INFO:2020-12-11_20:46:33.008:eval.py:120: model.ckpt-30699, predicted 2380/(2526) batches
INFO:2020-12-11_20:46:33.609:eval.py:120: model.ckpt-30699, predicted 2390/(2526) batches
INFO:2020-12-11_20:46:34.216:eval.py:120: model.ckpt-30699, predicted 2400/(2526) batches
INFO:2020-12-11_20:46:34.804:eval.py:120: model.ckpt-30699, predicted 2410/(2526) batches
INFO:2020-12-11_20:46:35.399:eval.py:120: model.ckpt-30699, predicted 2420/(2526) batches
INFO:2020-12-11_20:46:36.008:eval.py:120: model.ckpt-30699, predicted 2430/(2526) batches
INFO:2020-12-11_20:46:36.605:eval.py:120: model.ckpt-30699, predicted 2440/(2526) batches
INFO:2020-12-11_20:46:37.216:eval.py:120: model.ckpt-30699, predicted 2450/(2526) batches
INFO:2020-12-11_20:46:37.819:eval.py:120: model.ckpt-30699, predicted 2460/(2526) batches
INFO:2020-12-11_20:46:38.407:eval.py:120: model.ckpt-30699, predicted 2470/(2526) batches
INFO:2020-12-11_20:46:38.988:eval.py:120: model.ckpt-30699, predicted 2480/(2526) batches
INFO:2020-12-11_20:46:39.582:eval.py:120: model.ckpt-30699, predicted 2490/(2526) batches
INFO:2020-12-11_20:46:40.195:eval.py:120: model.ckpt-30699, predicted 2500/(2526) batches
INFO:2020-12-11_20:46:40.807:eval.py:120: model.ckpt-30699, predicted 2510/(2526) batches
INFO:2020-12-11_20:46:41.421:eval.py:120: model.ckpt-30699, predicted 2520/(2526) batches
INFO:2020-12-11_20:46:43.676:eval.py:67: model.ckpt-30699, accuracy=66.48775661637399, metric=0.19124932847848147, f1=0.19124932847848147
INFO:2020-12-11_20:46:43.676:eval.py:70: evaluation done, took 0:02:44.879326 s!
INFO:2020-12-11_20:46:43.676:eval.py:71: final_predictions saved to: /data1/hwt/deformer/data/predictions/bert/qqp-dev-predictions.json_

the prediction results of QQP dataset (file qqp-dev-predictions.json) are mostly 0

I wonder if this error just because I trained the model on GPU?

can you give me some advice , thanks !

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.