seems some device issue about docformer HOT 6 CLOSED

Definitely, that is an issue, which I was not able to solve, in the code itself, so here is how to solve it. You can copy-paste the model's code (i.e modelling.py code), into your py file, and then remove that specific import, and it works fine. I am trying to solve it, and I hope that it would be resolved in some time. Do let me know after you have tried it. Thanks,

from docformer.

These .cuda() calls are responsible for this error since they partially force some values to move to CUDA. I suggest we take all such calls out since the whole model can be moved if needed to the GPU using the .cuda().

from docformer.

@Jordy-VL Can you see it, if the updated PR solves the issue?

from docformer.

Hi, thanks for the update!
Well, I get another error now which might be introduced during the PR:

Traceback (most recent call last): File "example.py", line 31, in <module> v_bar, t_bar, v_bar_s, t_bar_s = feature_extractor(encoding) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/jordy/code/ICDAR/docformer/./src/docformer/modeling.py", line 512, in forward v_bar = self.visual_feature(image) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/jordy/code/ICDAR/docformer/./src/docformer/modeling.py", line 48, in forward x = self.resnet50(x) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward input = module(input) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward return self._conv_forward(input, self.weight, self.bias) File "/home/jordy/.virtualenvs/SOTA/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 384, 500] instead

from docformer.

Let me tell you the reason for this error.

Actually, the readme has been mentioned taking into consideration that, the create_features adds a dimension of 1 (in the earlier PR, the add_batch_dim argument was set to true, since at that time, we were not quite sure, how would we train it, so we decided to create individual element as a single batch, and then stack it, and do the training), however, now, I have been able to do the training without creating individual batch (just like any normal neural network training) and so the add_batch_dim argument is set to False, and would very soon update the readme for more appropriate training. If you wanna try you can just set the argument to true, and refer to this issue to help How to train docformer for custom dataset

Very soon, the training procedure and other utilities would be added.

Hope this helps,
Regards

from docformer.

added the "add_batch_dim=True" and now the example works :)

Thanks for the help!

from docformer.