Comments (2)
Hello Cybercandy,
You can currently use custom models as long as they follow the hugging face design. For example to use the brand new ALBERT implementation, you could do the following:
from transformers import AlbertTokenizer, AlbertModel
albert_model = AlbertModel.from_pretrained('albert-base-v1', output_hidden_states=True)
albert_tokenizer = AlbertTokenizer.from_pretrained('albert-base-v1')
custom_summarizer = Summarizer(custom_model=albert_model, custom_tokenizer=albert_tokenizer)
If you had a custom tuned model, you could use your own there. I do believe that the transformers library does have a pretrained CamemBERT model though.
from bert-extractive-summarizer.
Awesome, thanks @dmmiller612 for the quick reply - so in the case of German that is indeed included in the hugging face distribution I would go for:
from transformers import BertModel, BertTokenizer
bertgerman_model = BertModel.from_pretrained('bert-base-german-cased', output_hidden_states=True)
bertgerman_tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')
custom_summarizer = Summarizer(custom_model=bertgerman_model, custom_tokenizer=bertgerman_tokenizer)
from bert-extractive-summarizer.
Related Issues (20)
- unable to build on Mac m1 - Big Sur HOT 1
- ValueError: n_samples=4 should be >= n_clusters=40 HOT 1
- how to save model as pkl.file for deploying
- "from summarizer import Summaizer" HOT 1
- training custom model HOT 1
- can you please provide english.json file ,i was having issue that trainer folder is not there
- AWS Lambda + Container issue with model loading as /home is read only HOT 1
- Error when running xlnet for individual paragraphs on linux using gpu
- Reproducibility bug on run_embeddings method
- Don't load the SBERT model twice
- Which kind of model should I choose?
- How to use cached sentence embedding vector as the input instead of text?
- How to support Japaneses
- tensor size mismatch for specific input text
- TypeError: 'Summarizer' object is not callable
- Run Summarizer model on array of strings HOT 2
- Trying to mimic the API's result
- [News API] Summarization returns empty string HOT 2
- cannot import name summarizer HOT 1
- Need a way to force load on CPU when an unsupported GPU throws a pytorch error.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bert-extractive-summarizer.