jhart96 / keras_elmo_embedding_layer Goto Github PK

hi. greate code. but I have a question.
why are you using pre-padding while TensorFlowhub example uses post padding?
does not using pre-padding means Elmo model in tf-hub will ignore the last words?

Hi,
I am new into NLP and TF and wanted to use state of art embeddings into a simple news category classification problem. When I use your elmo.py code and try to build a LSTM model using it, it shows the following error:
'StaticHashTableV1' object has no attribute 'init'
This error is shown in this line:
self.lookup_table.init.run(session=K.get_session())

I checked it is related to initializing the table using tf.tables_initializer(). But I am getting the same error or maybe I am using it in wrong way
Kindly help
Thanks

@JHart96 @Deep1994

Please help me to overcome this issue below. Getting error at line no 3

I am using the below code in Tensorflow 2.0:

1. lstm_cell = tf.keras.layers.LSTMCell(units=128)
2. lstm_cell = tf.nn.RNNCellDropoutWrapper(lstm_cell,output_keep_prob=.5)
3. self._initial_state = lstm_cell.get_initial_state(128, tf.float32)

ValueError: slice index 0 of dimension 0 out of bounds. for strided_slice (op: StridedSlice) with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <0>, input[2] = <1>, input[3]

Hi
many thanks for your intersting example! I've got one question though:

You write in your Readme that it depends on the Tensorflow hub module. Could you elaborate that a bit more? Does that mean when building a model and storing it, it could only even be used with that said Tensorflow hub module? Or does that dependency only exist during the learning phase?

Cheers

Hi, I find an error when I run the code, the error is as follows:
InvalidArgumentError: slice index 1 of dimension 0 out of bounds.
[[Node: el_mo_embedding_2/module_apply_tokens/strided_slice_1 = StridedSlice[Index=DT_INT32, T=DT_INT32, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:GPU:0"](el_mo_embedding_2/module_apply_tokens/Shape/_173, loss/activation_1_loss/Mean/reduction_indices, el_mo_embedding_2/module_apply_tokens/aggregation/strided_slice_1/stack_1, loss/activation_1_loss/Mean/reduction_indices)]]
[[Node: el_mo_embedding_2/module_apply_tokens/bilm/RNN_1/RNN/MultiRNNCell/Cell1/rnn/stack/_361 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1671_.../rnn/stack", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op 'el_mo_embedding_2/module_apply_tokens/strided_slice_1', defined at:
File "C:\Users\Deep\Anaconda3\lib\site-packages\spyder\utils\ipython\start_kernel.py", line 245, in
main()
File "C:\Users\Deep\Anaconda3\lib\site-packages\spyder\utils\ipython\start_kernel.py", line 241, in main
kernel.start()
File "C:\Users\Deep\Anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "C:\Users\Deep\Anaconda3\lib\site-packages\zmq\eventloop\ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "C:\Users\Deep\Anaconda3\lib\site-packages\tornado\ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "C:\Users\Deep\Anaconda3\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "C:\Users\Deep\Anaconda3\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "C:\Users\Deep\Anaconda3\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "C:\Users\Deep\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "C:\Users\Deep\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 235, in dispatch_shell
handler(stream, idents, msg)
File "C:\Users\Deep\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "C:\Users\Deep\Anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 196, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "C:\Users\Deep\Anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "C:\Users\Deep\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2728, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "C:\Users\Deep\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2850, in run_ast_nodes
if self.run_code(code, result):
File "C:\Users\Deep\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2910, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
embed = ELMoEmbedding(idx2word=idx2word, output_mode="elmo", trainable=False)(main_input) # These two are interchangeable
File "C:\Users\Deep\Anaconda3\lib\site-packages\keras\engine\topology.py", line 617, in call
output = self.call(inputs, **kwargs)
File "H:\twitter_ADE\elmo.py", line 38, in call
return self.elmo_model(inputs, signature="tokens", as_dict=True)[self.output_mode]
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow_hub\module.py", line 203, in call
name=name)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow_hub\native_module.py", line 447, in create_apply_graph
restore_collections_predicate=(lambda key: key in import_collections))
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\training\saver.py", line 1955, in import_meta_graph
**kwargs)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\meta_graph.py", line 743, in import_scoped_meta_graph
producer_op_list=producer_op_list)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
return func(*args, **kwargs)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\importer.py", line 513, in import_graph_def
_ProcessNewOps(graph)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\importer.py", line 303, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3540, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3540, in
for c_op in c_api_util.new_tf_operations(self)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3428, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "C:\Users\Deep\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1718, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): slice index 1 of dimension 0 out of bounds.
[[Node: el_mo_embedding_2/module_apply_tokens/strided_slice_1 = StridedSlice[Index=DT_INT32, T=DT_INT32, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:GPU:0"](el_mo_embedding_2/module_apply_tokens/Shape/_173, loss/activation_1_loss/Mean/reduction_indices, el_mo_embedding_2/module_apply_tokens/aggregation/strided_slice_1/stack_1, loss/activation_1_loss/Mean/reduction_indices)]]
[[Node: el_mo_embedding_2/module_apply_tokens/bilm/RNN_1/RNN/MultiRNNCell/Cell1/rnn/stack/_361 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1671_.../rnn/stack", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

I don't know what is the wrong with my code, could you help me?

I am using the below code in Tensorflow 2.0:

1. lstm_cell = tf.keras.layers.LSTM(units=hidden_unit)
2. lstm_cell = tf.nn.RNNCellDropoutWrapper(lstm_cell,output_keep_prob=self.dropout_keep_prob)
3. self._initial_state = lstm_cell.get_initial_state(128, tf.float32)

I am getting an error at line 3

ValueError: slice index 0 of dimension 0 out of bounds. for strided_slice (op: StridedSlice) with input shapes: [0], [1], [1], [1] and with computed input tensors: input[1] = <0>, input[2] = <1>, input[3] = <1>.