Comments (3)
原始Transformer是指在用于NMT任务的Transformer吗?
from cal-flops-for-plm.
原始Transformer是指在用于NMT任务的Transformer吗?
嗯呢,有几个问题:
1.你的例子中是基于bert的,有一些参数,变量不太看得懂,比如from transformers import BertForSequenceClassification, BertTokenizer from transformers import RobertaForSequenceClassification, RobertaTokenizer
2.这个算法是在训练数据集,还是测试数据
3.模型的长度是自己给一个定长,还是实打实的算句子实际的长度
from cal-flops-for-plm.
1、BertForSequenceClassification、BertTokenizer、RobertaForSequenceClassification、RobertaTokenizer是import自huggingface-transformer,分别是用于句子分类的BERT模型、BERT分词器、用于句子分类的RoBERTa模型、RoBERTa分词器,具体代码,请参见 https://github.com/huggingface/transformers/tree/v2.5.1 ,本repo测试的transformer版本是2.5.1;
2、这个算法是在测试模型在计算过程中FLOPs,是与模型结构相关的性能,和使用数据集或者单条数据无关;
3、example.py中的长度是设置的一个最大长度,超过最大长度的部分会被Tokenizer截去,可以根据输入的情况进行更改。
from cal-flops-for-plm.
Related Issues (1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cal-flops-for-plm.