When I'm trying to generate the new midi file I'm receiving this error:
RUNTIME ERROR: bool value of Tensor with more than one value is ambiguous.
I decided to train the neural network with just 5 files before I sent it to my cluster. I was able to successfully generate the model and then I tried to generate some music with it. When I was running it through my debugger I successfully made it past line 47 ( result = mt(inputs, config.length, gen_summary_writer) ). I then print out the result and attempt to decode it. That is where it breaks. I have included the model if you see fit to test it yourself. I used the VScode debugger to see all the variables.
Command Line: "python generate.py -c config/base.yml config/generate.yml config/train.yml -m model" (my default is Python 3 and I'm working in python 3.7)

model.zip

Do you have any documents for validation NLL ? I am new to programming and learn from you.

Hello, �I have a question about the velocity. In processor.py file, you had a modified velocity which is calculated like modified_velocity = snote.velocity // 4 and when it comes to decode, it is replicated by velocity = event.value * 4. Why do you process velocity like that? And it caused a problem that the final velocity will be larger than 127. Thank you!

RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 10.73 GiB total capacity; 9.03 GiB already allocated; 946.69 MiB free; 221.61 MiB cached)

I get this error, possibly by of my machine. Any idea how to solve this?

hello jason,
thanks for your implementation! good job!
however, when i wanted to train the music transformer, i found that there is a mistake in the "train.py".
i received this error:
Traceback (most recent call last):
File "train.py", line 92, in
metrics = metric_set(sample, batch_y)
File "/home/mirlab2019/wanglei/MusicTransformer-pytorch/custom/metrics.py", line 69, in call
return self.forward(input=input, target=target)
File "/home/mirlab2019/wanglei/MusicTransformer-pytorch/custom/metrics.py", line 75, in forward
for k, metric in self.metrics.items()}
File "/home/mirlab2019/wanglei/MusicTransformer-pytorch/custom/metrics.py", line 75, in
for k, metric in self.metrics.items()}
AttributeError: 'tuple' object has no attribute 'to'

After further discovering, i found that the reason is that a tuple is returned in the line 91, train.py. And then the returned tuple is transferred to "metric_set()", however "metric_set()" should receive a tensor instead of a tuple. Actually, adding a new variable to receive the returned attention weights could avoid the above error.

Hi,

Thanks for the code. It gives me a base for mine.
I just tested the following implementation for the position embedding in custom/layers. It works faster so I offer you my solution. The difference in results due to numerical approx is < e^-12.

    pos_emb = np.zeros((max_seq, embedding_dim))
    for index in range(0, d_emb,2):
        pos_emb[:, index] = np.array([m.sin(pos/10000**(index/embedding_dim))
                                      for pos in range(max_seq)])
        pos_emb[:, index+1] = np.array([m.cos(pos/10000**(index/embedding_dim))
                                       for pos in range(max_seq)])
    return pos_emb

Hi, I have an error below,
NotADirectoryError: [Errno 20] Not a directory: 'model.py/save.yml'

when I run python train.py -c config/base.yml config/generate.yml config/train.yml -m model.py

(MusicTransformer-pytorch) hyeshinchu@gpu-2080ti-11:~/MusicTransformer-pytorch$ python train.py -c config/base.yml config/generate.yml config/train.yml -m model.py
Traceback (most recent call last):
File "train.py", line 20, in
config.load(args.model_dir, args.configs, initialize=True)
File "/home/hyeshinchu/MusicTransformer-pytorch/custom/config.py", line 88, in load
super().load(model_dir, configs, initialize, print)
File "/home/hyeshinchu/MusicTransformer-pytorch/custom/config.py", line 67, in load
self.save(model_dir)
File "/home/hyeshinchu/MusicTransformer-pytorch/custom/config.py", line 76, in save
with open(os.path.join(model_dir, self.CONFIG_FILE_NAME), "w+") as f:
NotADirectoryError: [Errno 20] Not a directory: 'model.py/save.yml'

Any comments are welcomed! Thanks in advance.

python train.py -c config/train.yml -m ./models

Traceback (most recent call last):
File "prj_dir/MusicTransformer-pytorch/custom/config.py", line 21, in getattr
return super().getattr(key)
AttributeError: 'super' object has no attribute 'getattr'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "prj_dir/MusicTransformer-pytorch/train.py", line 20, in
config.load(args.model_dir, args.configs, initialize=True)
File "prj_dir/MusicTransformer-pytorch/custom/config.py", line 89, in load
self._set_vocab_params()
File "prj_dir/MusicTransformer-pytorch/custom/config.py", line 92, in _set_vocab_params
self.pad_token = self.event_dim
File "prj_dir/MusicTransformer-pytorch/custom/config.py", line 23, in getattr
return self.dict[key]
KeyError: 'event_dim'

안녕하세요.

Music Transformer 페이퍼와 비교하면서 코들를 읽다가 질문이 있어서 올립니다.

1. 페이퍼 섹션 3.4 에 relative distance를 구하여 dot product 연산하는 부분이 있는데 코드에서는

self.E = torch.randn([self.max_seq, int(self.dh)], requires_grad=False)로 distribution을 쓰시더라구요.

이부분은 페이퍼와 다르게 하신건가요?

감사합니다.

Hi,

Would it be possible to obtain a pre-trained model? I keep running into either vanishing or exploding gradient problems when I run.

Thanks!

when I run the script like : sh dataset/scripts/ecomp_piano_downloader.sh dataset/midi
I get errors like :
No URLs found in -. rm: missing operand Try 'rm --help' for more information. rm: cannot remove ‘*’: No such file or directory

How can I fix it?

Hi, during training the model, the method _get_seq() in file data.py throws an IndexError. I think the reason is that the length of the given midi data is less than max_length.

Then I print the file name when this error happens and I found the midi file (after converted to pickle file) is Arciglione08.MID.pickle. The max_length is 2048 but the length of this data is 1777.

I download ecomp_piano as the input dataset. For all hyper-parameters I just use what is shown in the master branch and here is the command I used for training.
python train.py -c config/base.yml config/train.yml -m ../model

I think I could decrease the max_length to avoid this error. But could you please tell me what is the root cause of this error?

Thanks!

Hi, at your custom/layers.py, there are EncoderLayer and DecoderLayer.
you actually use EncoderLayer, which is solely self-attention.

in DecoderLayer, the forward function receives encode_out, which means DecoderLayer follows the vanilla transformer structure, I think.

In the Music Transformer paper, I remember that this uses only self-masked attention at decoder part.

Did you try using the vanilla transformer structure with relative attention for implementation?
How was the result compared to using only self-masked attention, if you tried?