hsuwanting / unified-summarization Goto Github PK

Thanks for the great work !

I am also trying to use the pertained model to evaluate the test set of CNN / Daily Mail summarization dataset. However, I keep having an error:
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value SentSelector/sent_encoder/bidirectional_rnn/bw/gru_cell/candidate/biases[[Node:SentSelector/sent_encoder/bidirectional_rnn/bw/gru_cell/candidate/biases/read = IdentityT=DT_FLOAT,_device="/job:localhost/replica:0/task:0/cpu:0"]]

I am running main.py using the following parameters:
python main.py --mode evalall --model end2end --vocab_path ./data/finished_files/vocab --data_path ./data/finished_files/test.bin --decode_method greedy --eval_method loss --log_root log --single_pass 1 --exp_name exp_sample --load_best_eval_model True --eval_ckpt_path ./log/end2end/exp_sample/train/bestmodel-51000

I am wondering if this is the correct parameters to run the pretrained model?

Your help is appreciated

Thanks!

Hi，
Is there a version of pytorch for unified model？

SAVE_PKL variable is passed as SAVE_PK which leads to an empty string being passed to a boolean flag.

Can i upgrade the code into tensorflow 2.0 version? By using tf.upgrade_v2?

Thank you for the great work!

I'm trying to use the pretrained model to decode my own preprocessed data (I do not need to evaluate it). However, I'm getting this exception when running main.py:

INFO:tensorflow:Loading checkpoint ./end2end/train/bestmodel-51000 INFO:tensorflow:Restoring parameters from ./end2end/train/bestmodel-51000 INFO:tensorflow:Failed to load checkpoint from ./end2end/train. Sleeping for 10 secs...

I figured this happens in util.py in load_ckpt(), when it gets to saver.restore(sess, "./end2end/train/bestmodel-51000")

I'm running main.py in the following way:
python2 main.py --mode eval --model end2end --vocab_path ./data/finished_files/vocab --data_path ./data/pgn/output/finished_files/chunked/test_* --decode_method greedy --eval_method loss --log_root . --pretrained_selector_path ./end2end/train/extractor_model --pretrained_rewriter_path ./end2end/train/abstracter_model --single_pass 1

So, my parameters are:
{'adagrad_init_acc': 0.1, 'batch_size': 16, 'beam_size': 4, 'convert_to_coverage_model': False, 'cov_loss_wt': 1.0, 'coverage': False, 'data_path': './data/pgn/output/finished_files/chunked/test_000.bin', 'decode_method': 'greedy', 'emb_dim': 128, 'eval_ckpt_path': '', 'eval_gt_rouge': False, 'eval_method': 'loss', 'exp_name': '', 'hidden_dim_rewriter': 256, 'hidden_dim_selector': 200, 'inconsistent_loss': True, 'inconsistent_topk': 3, 'load_best_eval_model': False, 'log_root': '.', 'lr': 0.15, 'max_art_len': 50, 'max_dec_steps': 100, 'max_enc_steps': 600, 'max_grad_norm': 2.0, 'max_select_sent': 20, 'max_sent_len': 50, 'max_train_iter': 10000, 'min_dec_steps': 35, 'min_select_sent': 5, 'mode': 'eval', 'model': 'end2end', 'model_max_to_keep': 5, 'pretrained_rewriter_path': './end2end/train/abstracter_model', 'pretrained_selector_path': './end2end/train/extractor_model', 'rand_unif_init_mag': 0.02, 'save_model_every': 1000, 'save_pkl': False, 'save_vis': False, 'select_method': 'prob', 'selector_loss_wt': 5.0, 'single_pass': True, 'start_eval_rouge': 30000, 'thres': 0.4, 'trunc_norm_init_std': 0.0001, 'vocab_path': './data/finished_files/vocab', 'vocab_size': 50000}

I'm using Tensowflow 1.1.0 and python 2.7.

• Is this the right way to configure the settings in order to decode my own documents?
• How to solve the exception issue?
  Any help is appreciated, thank you!

Hello, I failed to train the selector model. I used the command "sh scripts/selector.sh" , and then I got the error message:

{'adagrad_init_acc': 0.1,
'batch_size': 64,
'beam_size': 4,
'convert_to_coverage_model': False,
'cov_loss_wt': 1.0,
'coverage': False,
'data_path': 'data/finished_files/chunked/train_*',
'decode_method': '',
'emb_dim': 128,
'eval_ckpt_path': '',
'eval_gt_rouge': False,
'eval_method': '',
'exp_name': 'exp_sample',
'hidden_dim_rewriter': 256,
'hidden_dim_selector': 200,
'inconsistent_loss': True,
'inconsistent_topk': 3,
'load_best_eval_model': False,
'log_root': 'log',
'lr': 0.15,
'max_art_len': 50,
'max_dec_steps': 100,
'max_enc_steps': 600,
'max_grad_norm': 2.0,
'max_select_sent': 20,
'max_sent_len': 50,
'max_train_iter': 50000,
'min_dec_steps': 35,
'min_select_sent': 5,
'mode': 'train',
'model': 'selector',
'model_max_to_keep': 3,
'pretrained_rewriter_path': '',
'pretrained_selector_path': '',
'rand_unif_init_mag': 0.02,
'save_model_every': 10,
'save_pkl': False,
'save_vis': False,
'select_method': 'prob',
'selector_loss_wt': 5.0,
'single_pass': False,
'start_eval_rouge': 30000,
'thres': 0.4,
'trunc_norm_init_std': 0.0001,
'vocab_path': 'data/finished_files/vocab',
'vocab_size': 50000}
INFO:tensorflow:Starting selector in train mode...
max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/mnt/c/unified-summarization-master/batcher.py", line 408, in fill_example_queue
(article, abstract, extract_ids) = input_gen.next() # read the next example from file. article and abstract are both strings.
File "/mnt/c/unified-summarization-master/batcher.py", line 488, in text_generator
extract_ids_str = e.features.feature['extract_ids'].bytes_list.value[0]
IndexError: list index (0) out of range
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/mnt/c/unified-summarization-master/batcher.py", line 408, in fill_example_queue
(article, abstract, extract_ids) = input_gen.next() # read the next example from file. article and abstract are both strings.
File "/mnt/c/unified-summarization-master/batcher.py", line 488, in text_generator
extract_ids_str = e.features.feature['extract_ids'].bytes_list.value[0]
IndexError: list index (0) out of range

Operating Systems: Windows Subsystem for Linux (Ubuntu 18.04)

Do you know why? Thank you very much.

thanks for the codes! I have a question about the two-stages training as described in 3.5

I understand that you pretrained the extractor and abstractor and then fine-tune the abstractor by feeding in :
1)the predicted extracted sentences from extractor
2)ground truth abstract summary sentences(from dataset)

What happens if the predicted extracted sentences are not aligned with the ground truth abstract summary sentences? Is there some kind of matching?

or more importantly, what if the extracted sentences are incorrectly predicted, then wouldn't the fine tune process actually means we are "causing more damage" by minimizing the NLL of abstractor for a wrong target sequence?

Thanks in advance for answering!

I have a problem when I train the selector model.
The console always output recall:0, ratio:1, thres:0

Is this a normal phenomenon?

I'm trying to generate summaries for a different corpus using the end2end pretrained model provided. I've been able to do this for the cnn/dailymail corpus but haven't had any luck with another corpus. What's the easiest way to obtain the summaries from plain text files?

I've tried processing the data from my corpus by adding artificial highlights to my corpus since it's not provided. What I mean by this is that I've added a few @highlight at the end of each file of my corpus with its corresponding line.

Then in the make_datafiles.py I changed some things around to process my corpus and since I don't need any Rouge data I changed get_extract_summary to this.

def get_extract_summary(article_sents, abstract_sents):
return [], [], [], [], [], None

I run it using the end2end script. It's all left by default except these lines:
VOCAB_PATH='data/finished_files_PrePro_1/vocab' CKPT_PATH="models/unified_model/bestmodel-51000" MODE='evalall'
I'm using the vocab from the cnn/dailymail preprocessing. And ckpt_path is pointing to the provided pretrained model. finished_files_PrePro_1 is the cnn/dailymail preprocessed directory ( i use this vocab since I didn't generate a vocab preprocessing my corpus ) and finished_files has the processed corpus I am currently trying to summarize.

When I execute it I get an exception when loading the model:
INFO:tensorflow:Building graph...
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/tmp/batcher.py", line 421, in fill_example_queue
extract_ids = [int(i) for i in extract_ids]
ValueError: invalid literal for int() with base 10: ''

and then it gets stuck doing this:

WARNING:tensorflow:Bucket input queue is empty when calling next_batch. Bucket
queue size: 0, Input queue size: 0

I have a feeling I'm doing something I shouldn't in the preprocessing.

• Could you tell me how I should preprocess the corpus without highlights to decode my own corpus?

• It should be noted that my corpus only has around 500 files.

Thanks in advance.

Hi ,
When I reproduction your code, I found the result get from end2end model is intricate.
Beause, I think the end2end model can rewrite the article , then the output decoded summary can be different from the sentence in original article. But I find the decoded summary can be found in the original article.


I train the model follow your README file .
Thanks~

While running the below with with MODE='eval' for evaluation it throws error
sh scripts/eval_rewriter.sh

INFO:tensorflow:max_enc_steps: 400, max_dec_steps: 120
INFO:tensorflow:Loading checkpoint log/rewriter/exp_sample/train/model.ckpt-88000
INFO:tensorflow:Restoring parameters from log/rewriter/exp_sample/train/model.ckpt-88000
2019-01-31 10:14:56.590855: W tensorflow/core/framework/op_kernel.cc:1152] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for log/rewriter/exp_sample/train/model.ckpt-88000
2019-01-31 10:14:56.598651: W tensorflow/core/framework/op_kernel.cc:1152] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for log/rewriter/exp_sample/train/model.ckpt-88000
2019-01-31 10:14:56.601643: W tensorflow/core/framework/op_kernel.cc:1152] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for log/rewriter/exp_sample/train/model.ckpt-88000

Hi, thanks for your great work.

I get to know that there is an architecture difference between SummaRuNNer and extractor of this work.
In SummaRuNNer, they add absolute/relative positional embeddings to the classification layer, but in your work, you didn't add those embeddings (related code).
Is there any special reason for this architecture choice (e.g. faster training, performance degradation)? If so, can you tell it to me?

Thanks!

Hello,

I forked your project and adapted in to our own dataset.

There was no problem running selector and abstractor training.
However, when we run end2end training, we get this error for the selector.

Our pretrained_models + dataset

Not found: Key SentSelector/sent_encoder/bidirectional_rnn/fw/gru_cell/candidate/biases not found in checkpoint

We can only assume that there was something wrong with our dataset but were unable to find the cause. Do you have any suggestions?

Hello, i read your parper "A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss." recently.

I want to implement your model, but it's too hard to preprocess the CNN/Daily mail dataset.

Because there's no license about this repository, i want to know that i can use your code about preprocessing data.

This is one of output of the code,while the code is still running.
'''
Exception in thread Thread-134:
Traceback (most recent call last):
File "/home/test/anaconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/test/anaconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/test/WangHN/Unified Summarization/batcher.py", line 408, in fill_example_queue
(article, abstract, extract_ids) = input_gen.next() # read the next example from file. article and abstract are both strings.
File "/home/test/WangHN/Unified Summarization/batcher.py", line 488, in text_generator
extract_ids_str = e.features.feature['extract_ids'].bytes_list.value[0]
IndexError: list index (0) out of range
'''
What's wrong?

My teminal no error be found;
but the pretrained model cannot be found in the exp_sample directory;

I want to ask you, if I need to run more epoch, what should I do? Can I just add iteration directly? Because I used the same parameters as you, but my final result is still quite different from your results. How can I improve the accuracy of the model? My result is:

ROUGE-1:
ROUGE-1-F: 0.3784
ROUGE-1-R: 0.3895
ROUGE-1-P: 0.3759

ROUGE-2:
ROUGE-2-F: 0.1757
ROUGE-2-R: 0.1810
ROUGE-2-P: 0.1746

ROUGE-L:
ROUGE-L-F: 0.3599
ROUGE-L-R: 0.3705
ROUGE-L-P: 0.3576

Thank you for your help!

Hi，auther，
I'd like to ask you some questions，I ran around the server with the code you gave me on github, and the results were similar. But the reason you said abstracter higher than the unified model was because grandtruth was used for training and testing.
But shouldn't two stages train abstracter via threshold sentences？I don't understand this part, and in the final Table table-2, how did the result of your two-stage be produced, did you use the Abstracter Table in Github？How can I get it.
And the threshold you set it is 0.5, random or optimized?
Thanks for your help！

I think the higher scores in abstracter just for pretraining a bestmodel. And the best abstracter model is used in end2end, right? But the two stag's ROUGE score how to get ?

Is it possible to run this on Tensorflow Version >= 1.5 where CUDA 9 is supported ?

When I run evalall, the text in decoded and the text in reference do not point at the same article.
I checked that the data binaries are written correctly (article + @highlight + reference).

This problem occurs only with end2end training.

Do you have any suggestion?

Hello,

We are tyring to run your model with the pre-trained weights present and followed your steps in the read me file, but still got the error "IndexError : list index(0) out of range", in the lines 408 and 488 of bathcher.py file. Could you please explain the correct way of running it and help us out. Thank you so much !

Hello，I choose to start this code in windows，but I failed to train the selector model.
I run the settings as: "python main.py --model=selector --mode=train --data_path=data/finished_files/chunked/train_* --vocab_path=data/finished_files/vocab --log_root=log --exp_name=exp_sample --max_art_len=50 --max_sent_len=50 --max_train_iter=50000 --batch_size=64 --save_model_every=10 --model_max_to_keep=3"
the part of results shows:
INFO:tensorflow:Starting selector in train mode...
max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
Finished constructing vocabulary of 50000 total words. Last word added: reefhuis
I don't know why it can't continue to train the model，can you give me some suggestions? Thank you very much.

Hello,

I want to use your pretrained models to compare results. But I got the following error:

INFO:tensorflow:Loading best eval checkpoint
INFO:tensorflow:Loading checkpoint log/selector/exp_sample/eval/bestmodel-26580
INFO:tensorflow:Restoring parameters from log/selector/exp_sample/eval/bestmodel-26580
2019-04-03 13:28:48.933304: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key SentSelector/art_encoder/bidirectional_rnn/bw/gru_cell/candidate/bias
not found in checkpoint
INFO:tensorflow:Failed to load checkpoint from log/selector/exp_sample/eval. Sleeping for 10 secs...

I think this is because the parameters related to the model are in the checkpoint file. Can you upload this file?

Thank you very much!

; Input/output error
INFO:tensorflow:Failed to load checkpoint from log\end2end\exp_sample\train. Sleeping for 10 secs...

Did anyone meet the above problem while merely run sh end2end.sh? at this time, sinece mode='train' in thee script, how come loading?

Thank you for your great work!

I am confused about your evaluation method for the selector. The ratio is used as the evaluation method (The loss should be more appropriate as the evaluation method). Is there any special reason to use the ratio instead of the loss?

(line 174-182 of run_selector.py)
"""
# calculate running avg loss
running_avg_ratio = util.calc_running_avg_loss(ratio, running_avg_ratio, summary_writer, train_step, 'running_avg_ratio')

# If running_avg_loss is best so far, save this checkpoint (early stopping).
# These checkpoints will appear as bestmodel-<iteration_number> in the eval dir
if best_ratio is None or running_avg_ratio < best_ratio:
    tf.logging.info('Found new best model with %.3f running_avg_ratio. Saving to %s', running_avg_ratio, bestmodel_save_path)
    saver.save(sess, bestmodel_save_path, global_step=train_step, latest_filename='checkpoint_best')
    best_ratio = running_avg_ratio

"""

Traceback (most recent call last):
File "main.py", line 228, in
tf.app.run()
File "/home/test/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "main.py", line 164, in main
batcher = Batcher(FLAGS.data_path, vocab, hps, single_pass=FLAGS.single_pass)
File "/home/test/WangHN/Unified Summarization/batcher.py", line 351, in init
self._example_queue = Queue.Queue(self.BATCH_QUEUE_MAX * self._hps.batch_size)
TypeError: unsupported operand type(s) for *: 'int' and 'Flag'

Hi, When I use make_datafile.py, I find there are "pdb.set_trace()" in get_extract_summary function. I want to ask how to use it when the len(extract_sents) is 0, and what should I do to deal with len(extract_sents)==0, thanks.

Why is the model restored in each evaluation on the selector the model saved in the last iteration? So how to choose the best model in training?

Thanks for your great work, we are ongoing this text abstraction study.
However, none of the pretrained models download link is workable. Could you help to check it?

All the file have been downloaded but only with 1kb size.