Comments (2)
Hi @GenTxt ,
I ran a sampling using the same seed prompt. The results are as follows:
Winston sat back . A sense of complete helplessness had descended upon him . A deep breath of fear filled him . He was too late . In fact the last thing he had thought was the thought that he was doomed to this world , and no matter how soon , he would get to this . To escape this life , he was the only way to survive in this life . He wanted to be home , to find the others and return safely to earth , so that he could find the family of his great . Winston felt nothing wrong with his situation . He would be in no position to tell the other passengers at the boarding platforms that they would be rescued , and he could make certain that he never got to the ship . He would never take it to the safety of the vessel and not return . Once he had started his flight , he would be allowed to board the passenger compartment . Winston knew that if the ship got stuck in a mud lake , he would likely die there , which his life would be spared . He knew that he would not be able to escape the wreckage of ship without even getting through it or getting in the air currents of the mud . All of his thoughts had gone straight back to the day he had first dreamed up a life in this earth and so he would not ever regret this decision again . It was all just an accident and he didn ’ t need anything . So he was in a nominative position for the entire voyage on this ship but no one dared try anything in such a dire situation . He was just the one being held in his own , no matter how hard he tried . He had to be the one of the few people left that ever would see him go through this journey and get along with everyone . There would never be more sorrow on his shoulders to be brought back to life . Winston was still very angry that he had been sent over into the ocean for just this little time . He now felt that he should do nothing less to the fate of his children and siblings . Winston himself was not the kind of person to let anyone see how he felt . He was a young man , and he was a young orphan . Winston knew that he wasn ’ t the kind of person to put it off . It was just an accident and he was the type of person to take chances and make a mistake . He was too young for any of this to be his fault . There was no reason to blame his mistakes and he wanted to be on the ship , but he just couldn ’ t wait to start to be
We didn't implement the sampling function in the current huggingface unilm fork. Note that the sampled text would be different with different checkpoints. We use top-k (k=40) sampling algorithm to generate outputs. The flag --new_segment_ids
is necessary to initialize the correct segment embedding (index=2) for L2R LM. We also need to sample the distribution using the masked-LM way.
from unilm.
Hello again.
Still have issue trying to generate similar output comparable to your interesting example.
Ubuntu 18.04 using default 'unilmv1-large-cased.bin' from github link with following commands:
python3 decode_seq2seq.py --bert_model bert-large-cased --model_recover_path storage/unilmv1-large-cased.bin --new_segment_ids --mode l2r --input_file test.txt --max_seq_length 256 --max_tgt_length 128 --batch_size 16 --forbid_duplicate_ngrams --temperature 1.0 --length_penalty 0 --min_len 500 --top_k 40 --output_file test_unilm_mode-l2r-topk-min_length500.txt
test.txt is single line from paper:
Winston sat back. A sense of complete helplessness had descended upon him.
Result:
“ We ’ re going to be in the world ! ” He began to recount what he had learned . “ The world is a wreck , but here you stand in it , there are so many people who will not know it . And you are a damn coward , not to mention you are a little frightened , you are still scavenged . You are just a little unattracworthy that I ’ m going to take care of you . ” Worried that you were misunderstand , he shook his head . “ I mean , I have never taught you to learn the hard way , but if I
I've followed your instructions but can't find an option to "... sample the distribution using the masked-LM way."
Also, cannot find option to increase length of output comparable to your sample. --min_len 500 has no effect.
I assume I'm missing options in the above terminal commands?
Would appreciate if you could provide the missing options.
Cheers
from unilm.
Related Issues (20)
- The influence of t (temperature) in the E5 Model paper HOT 4
- (BEIT-3) I can't find transformers.py HOT 3
- Kosmos-2.5 Dependencies Should Be Updated HOT 1
- The pretraining code and model of beit3
- TROCR model weight file cannot be downloaded.
- BEIT3 RuntimeError: Size mismatch when loading checkpoint for BEiT3ForRetrieval model
- Kosmos-2.5 - Deployment Challenges on a Windows 11 + RTX 3090 PC HOT 1
- [BEIT-3] error happens when I evaluate BEiT-3 finetuned model on VQAv2 HOT 4
- Missing Documentation
- Kosmos 2.5 dataset availability
- Kosmos-2.5 - Image-to-markdown generation for images outside the sample-set provided is almost entirely garbled - output markdown is completely unusable.
- Kosmos-2.5 - Python version 3.10.x (and any other confirmed working versions) should be mentioned as a requirement to deploy & infer the Kosmos-2.5 model
- Kosmos-2 batch modality and processing speed
- how to resume training for layoutlmv3-publaynet
- TextDiffuser - SD2.1 code and weights
- the problem when load Beats model
- kosmos-2.5 | trying to connect to `openaipublic.blob.core.windows.net`
- layoutlmv3 pretrain
- Compile LayoutLMv3 using Neuron SDK for AWS Inferentia (inf1)
- Beit-3 , clarification on classification task and associated pretrained weights
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unilm.