Hey, I was able to load tensorflow checkpoints for both the pytorch-transformers v1.0.0 and the bertviz packaged version by editing modeling_bert.py as follows:
At line 105, change pointer = model to

        if len(name) == 1:                                                                                                                                                                                                                                                      
            if type(model).__name__ == "BertForSequenceClassification":                                                                                                                                                                                                         
                pointer = model.classifier                                                                                                                                                                                                                                      
            else:                                                                                                                                                                                                                                                               
                continue                                                                                                                                                                                                                                                        
        else:                                                                                                                                                                                                                                                                   
            pointer = model

Unfortunately, I got stuck here, see attached image. The "Layer" dialog box has no options in it, and there's no visualization nor error, just ...blank.

EDITED/SOLVED:

I was running the notebook from outside the bertviz directory by adding bertviz to the PYTHONPATH and following instructions from there. I can fix my issue by running the notebook in the directory directly, although I frankly don't know why-

__location__ = os.path.realpath(
        os.path.join(os.getcwd(), os.path.dirname(__file__)))
print(__location__)

shows the correct path regardless of where the notebook is executing from. Anyway, my issue was resolved. I hope this helps someone.

from bertviz.

Hi,
Thanks for reporting this. A couple of things:

• Did you try "num_labels" instead of "num_label"?
• There are a couple of related issues on pytorch_transformers: huggingface/transformers#676 and huggingface/transformers#580

If you're getting this error in BertForSequenceClassification, and you can't find a solution in the issues above, you might consider opening an issue with pytorch_transformers (https://github.com/huggingface/pytorch-transformers) as they would probably be in a better position to help.

from bertviz.

Thanks for reply.
I had tried both things "num_labels" and "num_label", getting the same error.
I have referred to huggingface/pytorch-transformers#676, but unable to resolve this.

from bertviz.

Thanks for sharing. BTW, the latest version of BertViz allows you to load a model directly using the transformers library, so this shouldn't be an issue going forward.

from bertviz.

@jessevig
I fine-tuned bert for a classification task. (following https://github.com/google-research/bert)
I mean to convert the tensorflow checkpoints to pytorch so that i can load the model into BertForSequenceClassification for visualization.

I used the following script

Initialise a configuration according to your model

config = BertConfig.from_pretrained('bert-base-uncased')
config.num_labels=4

You will need to load a BertForSequenceClassification model

model = BertForSequenceClassification(config)

tf_checkpoint_path = init_checkpoint

Load weights from tf checkpoint

load_tf_weights_in_bert(model, config, tf_checkpoint_path)

pytorch_dump_path = "./pytorch_bert_output"

Save pytorch-model

print("Save PyTorch model to {}".format(pytorch_dump_path))
torch.save(model.state_dict(), pytorch_dump_path)

I am getting the following error
BertForSequenceClassification object has no attribute 'bias'

Please help me out.
Thanks

from bertviz.

@chikubee

Hello, I try to visualize fine-tuned BERT model with BertModel and similar error occurs: "AttributeError: 'BertModel' object has no attribute 'bias"

This error occurs in file: "...Lib\site-packages\transformers\modeling_bert.py" line 104 and you can disable related lines in source codes.

Let me know if any questions.

from bertviz.