Can we get some samples? about megatron-lm HOT 3 CLOSED

Do you mean text output of the trained model? If so, you can see some sample output in the appendix of a paper we just put up on arXiv: https://arxiv.org/abs/1909.08053.

from megatron-lm.

Thanks, I see them.
So these are prompt-completions, correct? So it's GPT-2 on steroids haha?

Btw how well do you understand Transformers? I've spoke with many freelancers and collaborators and they seemed to not fully grasp it, none could re-code the Paper from scratch or explain every atom of the complete thing.

from megatron-lm.

Yes, for the samples in that paper GPT-2 is the underlying model (the code in this repo can also do BERT models).

After writing this code and training these models from scratch I think those of us working on this project can all say we have a pretty good understanding of transformers. :) All the information is there in the various papers, but it can take some determination to really understand it all. Now that there are several open source implementations studying the code is a good way to get to the nuts and bolts. Hopefully this code can help a bit!

from megatron-lm.