Dictionary and text corpus about ctcwordbeamsearch HOT 2 CLOSED

So my lexicon is restricted with words from my text corpus.

Yes, I believe that's right. (Unless you modify the code somehow).

Since the language model is built from the corpus of words, if a dictionary were included, the language model would not have information about any dictionary words not present in the corpus.

Of course there are standard methods to deal with this one could implement (i.e., smoothing, back-off, etc.) if it was essential; the code does seem to support smoothing for query ngrams not present in the training corpus, so perhaps it would not be too difficult to modify for your needs to add non-corpus words to the lexicon without using them in the bigram calculation.

But if you don't need a language model, you can just pass in a dictionary as a giant string of words using the "Words" option for the Scoring mode (lm_type).

from ctcwordbeamsearch.

Thanks for the detailed answer!

I'll use only a dictionary without a language model by now.

from ctcwordbeamsearch.