What's the log base of NgramLanguageModel.getLogProb() ? about berkeleylm HOT 5 OPEN

Answering my own question, it looks like it's 10.
It should probably be in the method description. Thanks!

Sorry, I thought I had replied.

That particular method doesn't actually know what base it is, since the LM was 
probably constructed from an ARPA file, and those files can be in whatever base 
they want (they are stored as logarithms). Building an LM with BerkeleyLM done 
in base 10 to mimic the behaviour of SRILM. So the answer is almost certainly 
"10", unless you constructed your LM in a non-standard way. 

Thanks! 
I still think it should be a part of the specification because the contract of 
an n-gram LM is a proper distribution p(w_i|...), and we can't have it without 
knowing the log base. It's true that many applications don't care about the log 
base, but some do (e.g., perplexity, text generation).

I glanced at the code, and it looks as though StupidBackoff is using log base 
e, while the Kneser-Ney models are using log base 10.  I've been using this 
package for my research, and it'd be nice to know what exactly the values are 
supposed to be.

Sorry, I missed this somehow.

I've added some comments in the latest SVN to hopefully clear this up. I'm not 
going to change it, just because I want to mimic SRILM in constructing 
Kneser-Ney LMs, and also don't want to change the logarithm base on 
StupidBackoffLms because that would change the models on people who are 
currently using them. Hope that clarifies things!