Comments (4)
The error message seems pretty informative - have you checked the length of your input samples?
from hedwig.
I did checked it, its of size 32,32
length tensor([[ 225, 1168, 377, 4, 274, 3939, 8, 54, 105, 268,
33, 5755, 83638],
[ 8, 54, 105, 19, 44, 44, 56528, 486, 767, 50,
5304, 1, 1],
[19369, 15135, 10210, 8251, 43371, 2545, 7481, 11348, 90504, 621,
56398, 1, 1],
[72795, 4372, 100, 70551, 1544, 1034, 3004, 13160, 6, 1370,
1, 1, 1],
[19919, 53218, 80, 1002, 1115, 8, 54, 105, 3386, 19796,
1, 1, 1],
[ 2, 69127, 1703, 3, 1077, 59162, 16, 1174, 50016, 88044,
1, 1, 1],
[ 2, 6726, 67871, 4653, 11485, 83, 6732, 83, 633, 38001,
1, 1, 1],
[87454, 20624, 4879, 4357, 430, 80, 6, 6026, 2196, 1489,
1, 1, 1],
[36913, 277, 286, 13804, 31, 14327, 6, 36907, 4260, 1,
1, 1, 1],
[ 9, 15352, 95585, 2684, 72, 5433, 486, 24, 47224, 1,
1, 1, 1],
[ 5564, 299, 10120, 58, 5164, 7374, 84, 4070, 70076, 1,
1, 1, 1],
[ 14, 2, 3648, 3, 519, 3637, 36810, 1, 1, 1,
1, 1, 1],
[ 519, 6904, 10666, 32768, 74735, 48988, 16957, 1, 1, 1,
1, 1, 1],
[83525, 38609, 441, 63150, 557, 50244, 856, 1, 1, 1,
1, 1, 1],
[ 8, 54, 105, 6929, 74, 10401, 36925, 1, 1, 1,
1, 1, 1],
[36812, 21, 1872, 261, 121, 76565, 27, 1, 1, 1,
1, 1, 1],
[ 66, 1099, 971, 6, 11, 4987, 1, 1, 1, 1,
1, 1, 1],
[73323, 1517, 2818, 30047, 19, 80137, 1, 1, 1, 1,
1, 1, 1],
[ 111, 1034, 69019, 36800, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[94708, 20, 19813, 1470, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[46071, 58952, 76, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[37106, 75713, 232, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[ 65, 83533, 36175, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[36828, 428, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[38424, 58372, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[37562, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[94372, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[36801, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[36958, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1],
[ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1]], device='cuda:0') tensor([13, 11, 11, 10, 10, 10, 10, 10, 9, 9, 9, 7, 7, 7, 7, 7, 6, 6,
4, 4, 3, 3, 3, 2, 2, 1, 1, 1, 1, 0, 0, 0],
device='cuda:0')
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/ubuntu/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/ubuntu/new_mount_device/hedwig/models/reg_lstm/__main__.py", line 149, in <module>
trainer.train(args.epochs)
File "/home/ubuntu/new_mount_device/hedwig/common/trainers/classification_trainer.py", line 93, in train
dev_acc, dev_precision, dev_recall, dev_f1, dev_loss = self.dev_evaluator.get_scores()[0]
File "/home/ubuntu/new_mount_device/hedwig/common/evaluators/classification_evaluator.py", line 40, in get_scores
scores = self.model(batch.text[0], lengths=batch.text[1])
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/new_mount_device/hedwig/models/reg_lstm/model.py", line 74, in forward
x = torch.nn.utils.rnn.pack_padded_sequence(x, lengths, batch_first=True)
File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/utils/rnn.py", line 244, in pack_padded_sequence
_VF._pack_padded_sequence(input, lengths, batch_first)
RuntimeError: Length of all samples has to be greater than 0, but found an element in 'lengths' that is <= 0
from hedwig.
I see that the last three elements have values 0, 0, 0. Even though your input is of non-zero length, the length vector might not have been set properly. Could that be the issue?
from hedwig.
I see that the last three elements have values 0, 0, 0. Even though your input is of non-zero length, the length vector might not have been set properly. Could that be the issue?
Yeah I felt the same. But
when I run below works fine
python -m models.bert --dataset Reuters --model bert-base-uncased --max-seq-length 256 --batch-size 16 --lr 2e-5 --epochs 30
but when I run
python -m models.reg_lstm --dataset Reuters --mode static --batch-size 32 --lr 0.01 --epochs 30 --bidirectional --num-layers 1 --hidden-dim 512 --wdrop 0.1 --embed-droprate 0.2 --dropout 0.5 --beta-ema 0.99 --seed 3435
So I am thinking where the error might be, because of LSTM or so?
from hedwig.
Related Issues (20)
- Reg-LSTM HOT 1
- Calling BertTokenizer.from_pretrained() with the path to a single file or url is deprecated HOT 1
- Pretrained weights via model zoo HOT 2
- [DocBERT] Can DocBERT bin any length document without truncaiton? HOT 1
- DocBERT HOT 3
- Alternate hosting for hedwig-data HOT 8
- Fine-tuning BERT for a new dataset
- Bert Inference Issue HOT 1
- Where I can get the KD model
- does DocBERT freeze all BERT layers and add a fully connected layer on the top for classification?
- Killed message on trying models.reg_lstm HOT 1
- Getting AttributeError: Can't get attribute 'gelu' on <module 'transformers.modeling_bert' in "/hedwig/models/bert/__main__.py", l
- Getting impossible predicted labels (all zeroes) from custom data
- [Question] What are the "robust" datasets?
- labels of the data
- Paper of HBERT HOT 1
- Scaled embedded dropout mask HOT 1
- BUG report
- UnpicklingError with different BERT models
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hedwig.