mesnilgr / iclr15 Goto Github PK

Alpha: 0.000002 Progress: 100.03% Words/thread/sec: 20.90k
real 182m22.244s
user 422m39.620s
sys 0m22.516s
../iclr15/scripts/paragraph.sh: line 16: ../liblinear-1.96/train: No such file or directory
../iclr15/scripts/paragraph.sh: line 17: ../liblinear-1.96/predict: No such file or directory
tail: cannot open ‘out.logreg’ for reading: No such file or directory
../iclr15/scripts/paragraph.sh: line 26: ../liblinear-1.96/train: No such file or directory
../iclr15/scripts/paragraph.sh: line 27: ../liblinear-1.96/predict: No such file or directory
tail: cannot open ‘out.logreg’ for reading: No such file or directory
rm: cannot remove ‘model.logreg’: No such file or directory
rm: cannot remove ‘out.logreg’: No such file or directory
Cloning into 'nbsvm'...
The authenticity of host 'github.com (192.30.252.128)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list of known hosts.
Connection closed by 192.30.252.128
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
../iclr15/scripts/nbsvm.sh: line 2: cd: nbsvm: No such file or directory
python: can't open file 'nbsvm.py': [Errno 2] No such file or directory
tail: cannot open ‘../scores/NBSVM-VALID’ for reading: No such file or directory
mv: cannot move ‘tmp’ to ‘../scores/NBSVM-VALID’: No such file or directory
python: can't open file 'nbsvm.py': [Errno 2] No such file or directory
tail: cannot open ‘../scores/NBSVM-TEST’ for reading: No such file or directory
mv: cannot move ‘tmp’ to ‘../scores/NBSVM-TEST’: No such file or directory

['RNNLM'] valid / test 83.54 / 86.16
/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py:816: UserWarning: loadtxt: Empty input file: "scores/PARAGRAPH-VALID"
warnings.warn('loadtxt: Empty input file: "%s"' % fname)
Traceback (most recent call last):
File "./../iclr15/scripts/combine.py", line 65, in
valid, test = load("VALID", c), load("TEST", c)
File "./../iclr15/scripts/combine.py", line 24, in load
data = normalize(data[:, 2] - 0.5)
IndexError: too many indices
ubgpu@ubgpu:/github/iclr15$
ubgpu@ubgpu:/github/iclr15$

word2vec.c takes train.txt and outputs vocab.txt and vectors.txt which is later used to create sentence_vectors.txt for my case. But I am facing trouble converting my input sentence in vector format as required for classification by loading the vocab and train files in word2vec.c. Quick reply is appreciated. Thanks!

try to fix it to
https://www.csie.ntu.edu.tw/~cjlin/liblinear/oldfiles/liblinear-1.96.zip
this will be ok

Hi Grégoire,

First of all, thank you for making your scripts available, It is much appreciated :)

I used your provided code to replicate the results reported in the ensemble paper:
http://arxiv.org/abs/1412.5335

I noticed that the reported accuracy of 89.3% using solely the paragraph vectors is not achievable on the test set, and is actually what I'm getting on the validation set. All other scores appear to be correct.

Below is the final output of your code, ran in full, without any modifications:

['RNNLM'] valid / test 83.32 / 86.38
['PARAGRAPH'] valid / test 89.38 / 88.94
['NBSVM'] valid / test 91.02 / 91.87

['RNNLM', 'PARAGRAPH'] valid / test 89.86 / 90.32 - weigths = [ 0.48245512 0.51754488]
['RNNLM', 'NBSVM'] valid / test 90.76 / 92.13 - weigths = [ 0.47791671 0.52208329]
['PARAGRAPH', 'NBSVM'] valid / test 91.46 / 92.21 - weigths = [ 0.49545455 0.50454545]

['RNNLM', 'PARAGRAPH', 'NBSVM'] valid / test 91.32 / 92.47 - weigths [ 0.31594115 0.33892007 0.34513878]

As you see, the performance of paragraph vectors on the test set appears to be lower than what's reported on the paper. I do, however, get an accuracy of ~89.3% on the validation set.

I repeated the experiments a few times, to make sure this was not due to a random factor (vector initialization/order of SGD updates from multiple threads etc.), but there was no significant variation on accuracy scores.

Just for my own sanity, could it be that you're reporting the wrong number on the iclr paper (validation, instead of test acc.)? Thanks a lot :)

Best regards,
Stefanos

Does -sentence-vectors 1 option to word2vec switch on Paragraph Vector mode?