How to train on Google n-grams about berkeleylm HOT 3 CLOSED

Hi, Joseph, thanks for your question.

The format should be as in the example directory: 
test/edu/berkeley/nlp/lm/io/googledir

Specifically, the directory look like

# 1gms/vocab_cs.gz [here, vocab_cs.gz should have the unigram frequencies 
sorted in decreasing order of frequency]
# 2gms/2gm-0001.gz 2gm-0002.gz …
# 3gms/3gm-0001.gz 3gm-0002.gz … 
# ...

Given your directory structure, you will need create an [n]gms directory for 
n=1..5, and then copy/soft-link all files for each order to the corresponding 
[n]gms directory. You might also need to create the vocab_cs.gz by sorting the 
unigram file, though this comes with at least the English distribution (in 
1gms). 

I have added additional documentation about this to the example script for the 
next release. 

Thanks, that worked great.