I have trained the model on LJ Speech for around 670k steps. When using python synthesize.py
command I have received this error.
Graph loaded
2018-04-25 19:19:45.131340: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-04-25 19:19:45.188585: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-04-25 19:19:45.188789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
totalMemory: 10.91GiB freeMemory: 114.62MiB
2018-04-25 19:19:45.188802: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-04-25 19:19:45.189340: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 114.62M (120193024 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
Restored!
0%| | 0/200 [00:00<?, ?it/s]Exception KeyError: KeyError(<weakref at 0x7fc729ad8100; to 'tqdm' at 0x7fc729b2e350>,) in <bound method tqdm.__del__ of 0%| | 0/200 [00:00<?, ?it/s]> ignored
Traceback (most recent call last):
File "synthesize.py", line 69, in <module>
synthesize()
File "synthesize.py", line 59, in synthesize
_y_hat = sess.run(g.y_hat, {g.x: texts, g.y: y_hat, g.ref: ref})
File "/home/m/anaconda3/envs/ttse/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/m/anaconda3/envs/ttse/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (16, 188) for Tensor u'Placeholder:0', which has shape '(32, 188)'