chrisjmccormick / word2vec_commented Goto Github PK

Hi , I read over your comment on the subsampling part.
The c implementation is
(sqrt(vocab[word].cn / (sample * train_words)) + 1) * (sample * train_words) / vocab[word].cn

However, in your comment, you says

Using the default 'sample' value of 0.001, the equation for ran is:
ran = (sqrt(x / 0.001) + 1) * (0.001 / x)

Should it be ran = (sqrt(x / 0.001 *train_words ) + 1) * (0.001 * train_words / x) instead ?

Thank you

Hey, I was just going through your code, and I think it's great! Just a minor correction here.
The Formatting of the markdown for the header seems to be off.

Regards

In word2vec.c file，it seems that variabe d is not declared.

The comment says that for word 'hat'

(((h * 257) + a) * 257 + t)

but the code has hash initialized to 1 which makes the comment wrong

By the way thank you very much for comments sir. 👍

Hi ,
First of all, Thanks for the great post and commentary. The example in the word2phrase.c - getWordHash function seems wrong to me.

Line 108 in word2phrase.c

hash = ((((h * 257) + a) * 257) + t) % 30E6

i guess the correct version should be :

hash = (((((hash * 257) + h) * 257) + a)*257+t) % 30E6

Please correct me if i am wrong.

In the comment in line-1547, don't you think it should be sigmoid instead of softmax?

Dear Chris

I came across papers saying word2vec (SGNS) weighs context word different based on the distance to the centered word (higher weight for closer ones). I am quite curious on that and want to check their code, but had a hard time to figure this out...

I find your word2vec repo with many great comments. If possible, could you please point me/make a comment on where that weighing scheme is implemented in word2vec.c? (Or maybe they did not implement this at all...)

Thanks,
Matt

I think there's a small mistake in your writeup.

So, we do not require all sentences to be separated by two newline characters. Since we unget(ch) whenever ch == '\n', a single newline character will suffice. That is, when we encounter a single newline character, we put it back. In the next invocation of ReadWord(), we start at a=0 and immediately find our old newline character, so we return the EOS token.

Therefore, it suffices for every sentence to be on a line by itself.

I usually add a space to the end of the line before adding the newline character, but that's just my personal preference.

Let me know if I'm wrong, but this was my understanding of the code.