pclucas14 / gansfallingshort Goto Github PK

The generators of different models are not the same in layers and dimensions, why not keep the generators sharing the same model size as the oracle model just like the conventional practice?

Hi,

Sorry one more question,
Which arg should I follow to train SEQGAN(not leakGAN)?
Is it the one in real_data_experiments/trained_models/news/word/best_gan/args.json?
Note that "seqgan_reward": 0, so I guess that's for leakGAN?

Thanks!

There is no best_mle in trained_models/news/word/. Would you please provide the best_mle , thanks very much.

Hi,
I found a problem with the training data in Table 5 in your paper. COCO's selfbleu5 is bigger than selfbleu4. Here is the BLEU and SELF-BLEU I tested on test data of COCO.
bleu2=0.744 bleu3=0.526 bleu4=0.344 bleu5=0.222
self-bleu2=0.895 self-bleu3=0.751 self-bleu4=0.581 self-bleu5=0.422

I ran score_models.py to evaluate the lm and rlm, loading the trained_models/news/word/best_mle/models/gen80th model. But the rlm results I got were worse than those in your best_mle_temp_rlm_score.csv.
Are the parameter settings during rlm training incorrect? I used the parameter settings of get_rlm_args in your code and args.mle_epochs=80.
I have measured the rlm when alpha=1.0 and the result is rlm=4.02, which is much worse than 3.991.
Thanks.

Hi, I like the paper very much, and recommended it to people in my group, thanks for the good work.
Question about the code, it seems that you implemented CoT, I try to run it using the real_data_experiments/trained_models/news/word/best_CoT_nlltest/args.json, however there's an error:

File "main.py", line 203, in main
avg_accs += [(fake_acc+real_acc)/2]
UnboundLocalError: local variable 'fake_acc' referenced before assignment

I guess fake_acc should not be calculated when training for CoT. Is the code ready for running CoT or does it need revision?
Thanks!

According the Self-BLEU original paper Zhu et al. 2018, each generation is compared against all the other references.

The current Self-BLEU implementation includes the selected hypothesis in the list of references. This risks inflation in the self-bleu scores as there will be always a direct match between the hypothesis and one of the references.

    def get_bleu(self):
        ngram = self.gram
        bleu = list()
        reference = self.get_reference()
        weight = tuple((1. / ngram for _ in range(ngram)))
        with open(self.test_data) as test_data:
            for hypothesis in test_data:
                hypothesis = nltk.word_tokenize(hypothesis)
                bleu.append(nltk.translate.bleu_score.sentence_bleu(reference, hypothesis, weight,
                                                                    smoothing_function=SmoothingFunction().method1))
        return sum(bleu) / len(bleu)

I understand that this is the implementation of the Texygen as is but I was wondering if
we should remove the target hypothesis from the set of references or am I missing something here?

Thanks for the help in advance

While using best_mle parameters, I found this code also use GAN for training.