Thank you for this excellent example. I am trying to implement my model which is similar to this with the tensorflow 1.2. But there is something wrong:
InvalidArgumentError (see above for traceback): Saw a non-null label (index >= num_classes - 1) following a null label, batch: 0 num_classes: 218 labels: 58,191,215,196
[[Node: loss/CTCLoss = CTCLoss[ctc_merge_repeated=true, ignore_longer_outputs_than_inputs=false, preprocess_collapse_repeated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](fc/transpose/_163, _arg_Placeholder_3_0_3, _arg_Placeholder_2_0_2, _arg_Placeholder_4_0_4)]]
[[Node: loss/CTCLoss/_165 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_4527_loss/CTCLoss", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]]
This is my code:
graph = tf.Graph()
with graph.as_default():
# feature inputs e.g. fbank, MFCC
# size : [batch_size, timesteps, feature_dim]
inputs = tf.placeholder(tf.float32, [None, None, FLAGS.n_mfcc])
# use sparse_placeholder to generate a SparseTensor required by ctc_loss op
targets = tf.sparse_placeholder(tf.int32, [None])
.....
# Time major
logits = tf.transpose(logits, (1, 0, 2))
tf.summary.histogram('fc', W)
tf.summary.histogram('fc', b)
with tf.name_scope('loss'):
loss = tf.nn.ctc_loss(targets, logits, seq_len)
with tf.name_scope('cost'):
cost = tf.reduce_mean(loss)
....
for i, batch in
enumerate(datagen.iterate_train(FLAGS.batch_size, shuffle=shuffle,
sort_by_duration=sortagrad)):
train_inputs = batch['x']
train_targets = sparse_tuple_from(batch['y'])
train_seq_len = batch['label_lengths']
feed = {inputs: train_inputs,
targets: train_targets,
seq_len: train_seq_len}
batch_cost, _ = session.run([cost, optimizer], feed)
train_cost += batch_cost*FLAGS.batch_size
train_ler += session.run(ler, feed_dict=feed)*FLAGS.batch_size
The labels seems to be considered as one label together, how can I fix this issue?
Many thanks.