davidsvy / transformer-xl Goto Github PK

First off, thank you so much for making your implementation open source. It's a huge help!

I'd like to use the Gated Transformer-XL implementation for a time series RL project of mine and would like to better understand what parts of the code are necessary to benefit from GTrXL. Specifically, I see the mem_list values, but didn't see that mentioned in this explanation of GTrXL. I'm still fairly new to ML, so I'm having a hard time understanding if it'd be okay to remove the mem_list part of the code. Or is it a core part of GTrXL?

Thank you for this work. Will you provide an implementation of the Pytorch version in the future?

Hello,

Thank you for sharing nice code. I finished training model using 'python train_music.py' command. I can check several checkpoint file was generated under 'checkpoints_music' folder.

Then, I tried to load one of them to generate music using 'python generate_music.py 1 /media/kimbring2/Steam/transformer-xl/checkpoints_music/optimizer1.npy' and met error of title.

I am not sure what is a problem. How can I solve that problem?

It is also good idea you share one of checkpoint of your side.

Thank you

Hello, thank you so much for sharing your work in this space. This has been very helpful for me to grok the transformer-xl architecture beyond a simple toy app. I am trying to generate music using the steps provided in the readme however when attempting to run the generate_music.py file I'm getting the above error. I've tried searching throughout the repo to align these layers but I haven't been able to fix it. Any help in resolving this error is greatly appreciated.