fail k2.compose(H, LG, inner_labels="tokens") about k2 HOT 13 OPEN

Can you clarify the size of your H and LG? I suspect it is an overflow of int32.

from k2.

Similar to #1010. Any way to solve this?

from k2.

2024-05-10 03:23:26,208 INFO [compile_hlg.py:133] H shape before Composing HLG: (501, None)
2024-05-10 03:23:26,208 INFO [compile_hlg.py:134] LG shape before Composing HLG: (11745505, None)

from k2.

Similar to #1010. Any way to solve this?

That's true, Smaller LG Compose succeeded （H.shape: 501, LG.shape: 11745505）, the larger LG Compose fail （H.shape: 501, LG.shape: 51395348）. the larger LG there is no other good way to Compose. how much is the upper limit of Compose

from k2.

If this is a graph-building exercise of some kind it may be better to use the kaldifst project https://k2-fsa.github.io/kaldifst/
to do the graph-building, then if needed it can be converted to the k2 format.

from k2.

If this is a graph-building exercise of some kind it may be better to use the kaldifst project https://k2-fsa.github.io/kaldifst/ to do the graph-building, then if needed it can be converted to the k2 format.

I've tried KaldiFST following the PR here, but the result just did not change.

from k2.

What do you mean "just did not change"?

from k2.

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

from k2.

You mean you can not compile HLG even with openfst?

By using openfst, you should convert H and LG from k2 to openfst, then compile HLG using openfst, convert the HLG back to k2 at last.

from k2.

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

Could you show the error logs after switching to kaldifst?

from k2.

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

Could you show the error logs after switching to kaldifst?

Sorry that I didn't save the logs. I followed the LibriSpeech script in egs/librispeech/ASR/local/compile_hlg_using_openfst.py.

It started running slowly with openfst for several hours, and the same error as mentioned in this issue happened again when composing HLG at Line-156.

My lm.arpa has 80 million grams with order 1 to 3.

from k2.

openfst

k2 to openfst use "from kaldifst.utils import k2_to_openfst ---> H_fst = k2_to_openfst(H)". compose use "HLG_fst = kaldifst.compose(H_fst, LG_fst)"， it success. how to convert openfst to k2

from k2.

openfst

k2 to openfst use "from kaldifst.utils import k2_to_openfst ---> H_fst = k2_to_openfst(H)". compose use "HLG_fst = kaldifst.compose(H_fst, LG_fst)"， it success. how to convert openfst to k2

Check files of k2-fsa/icefall#606. The conversion between k2 and fst is successful, but I guess the problem here is that k2.compose is still used after converting fst to k2.

from k2.