Comments (3)
Self-reply to the first part: you need to compress both files together, like:
tar cvzf bert_model.tar.gz bert_config.json pytorch_model.bin
I still get the "weights not initialized" message though.
from bert-toxic-comments-multilabel.
Sel-reply to the second part: looking closely at the error it makes sense: it states that it is missing weights for the classifier layers at the end of the model. These layers have been added on top of the Bert encoder, so obviously we don't have pre-trained weights for them.
A bit more disconcerting is another warning complaining that some of the weights in the source have not been used. Hopefully these correspond to the final Bert layers in the original model (those that have been swapped by the new classifier layers: it makes sense that we lack weights for the new layers while we are not using our weights for the discarded layers).
For the record, this is the list of unused layers:
'cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias'
And this is the list of layers with no pre-trained weights:
'classifier.weight', 'classifier.bias'
from bert-toxic-comments-multilabel.
I met this issue too. The problem is might I download the Tensforflow version pre-train model, instead of the pytorch's version. Here is the source code.
Download the model from below links, which contains the .bin
file:
PRETRAINED_MODEL_ARCHIVE_MAP = {
'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz",
'bert-large-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased.tar.gz",
'bert-base-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased.tar.gz",
'bert-base-multilingual': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual.tar.gz",
'bert-base-chinese': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-chinese.tar.gz",
}
from bert-toxic-comments-multilabel.
Related Issues (17)
- Change `PreTrainedBertModel` to `BertPreTrainedModel` HOT 2
- Wrong number of labels
- Unable to freeze/unfreeze HOT 1
- Is there a way to do the fitting on an already trained model?
- Does some one know the Segmentation fault (core dumped)
- FileNotFoundError: [Errno 2] No such file or directory: 'BERT\\pytorch_model.bin' HOT 1
- Target size is not matching with the input size HOT 1
- Re: Installation of apex
- AttributeError: 'BertForMultiLabelSequenceClassification' object has no attribute 'module' HOT 3
- What is in the classes.txt file? HOT 1
- Confused details about label_ids HOT 2
- Tips how to use this for multi-class multi-label classification
- Issue in importing Apex HOT 6
- CUDA out of memory. What can I do to improve model performance? HOT 3
- What is in the 'val.csv' file?
- FileNotFoundError: File val.csv does not exist HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bert-toxic-comments-multilabel.